Re: [GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs

2017-04-20 Thread Otto Fowler
Maybe it is the relative parent path? On April 20, 2017 at 20:05:31, mmiklavc (g...@git.apache.org) wrote: Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/538 Further, the reason that install worked is not because it needs the artifacts in the

Re: Reducing Warnings in Build

2017-04-21 Thread Otto Fowler
Until there is a use case supporting changing to other then the 80% plus case, this should not be changed. or a customer requirement etc. This would need some design and discussion time as well to map out all the implications for the complete pipeline. I think configurable charset should be

Re: Failing build

2017-04-20 Thread Otto Fowler
Maybe it is a container thing? I don’t think my PR for extensions has the problem, or has had it in my travis - and it is a vm in travis. On April 20, 2017 at 09:06:21, Casey Stella (ceste...@gmail.com) wrote: oh man, yeah, it's pissing me off too. I'm still looking at ways to get around

Re: [DISCUSS] next release proposal

2017-04-20 Thread Otto Fowler
+1 on the proposed changes. I think that it is important that if it is in reach, we ‘complete’ any major theme of the release. In this case kerberos. I think that given where pycapa and rest are, we should make the effort to get them in. On April 20, 2017 at 08:52:11, Ryan Merriman

Re: profiler Syntax error

2017-08-02 Thread Otto Fowler
Can you try the rule without ‘in’ as a key? “in” is a reserved word in Stellar. What that error means is that the ANTLR lexer and parser cannot figure out what the statement means because of ambiguity, most likely introduced by the use of a keyword. (in is not a valid statement. On August 2,

Re: profiler Syntax error

2017-08-02 Thread Otto Fowler
: TO_UPPER(in) due to: org.apache.metron.stellar.dsl.ParseException: Syntax error @ 1:9 no viable alternative at input 'TO_UPPER(in' On August 2, 2017 at 17:12:27, Otto Fowler (ottobackwa...@gmail.com) wrote: Can you try the rule without ‘in’ as a key? “in” is a reserved word in Stellar. Wha

SimpleEnrichmentFlatFileLoaderIntegrationTest …..

2017-08-03 Thread Otto Fowler
Results : Failed tests: SimpleEnrichmentFlatFileLoaderIntegrationTest.testLocalLineByLine_gz:362 expected:<1000> but was:<991> Is there a reason why this seems to be happening more often now?

Re: [DISCUSS] Code Reformatting next steps

2017-08-11 Thread Otto Fowler
What do we do for PR’s? I don’t think I should run this on 777 for example while it is under review. But, I don’t think running it as a last commit before landing is correct either. Will it be: PR as is ( if you haven’t reformatted, don’t ) Follow on PR with only reformatting ?? On August

Re: [DISCUSS] Code Reformatting next steps

2017-08-11 Thread Otto Fowler
We should add something to the PR template about this. Did we change the contributor’s guide? Also, are there common steps to take with Intellij, so that we all do the same thing for mass reformats? On August 11, 2017 at 08:14:21, Justin Leet (justinjl...@gmail.com) wrote: Now that METRON-746

Metron Alerts bombing in Travis?

2017-08-11 Thread Otto Fowler
Has anyone seen this error in travis? I am going to delete my caches and try again : [ERROR] npm ERR! Linux 4.4.0-83-generic [ERROR] npm ERR! argv "/home/travis/build/ottobackwards/metron/metron-interface/metron-alerts/node/node"

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Otto Fowler
Deleting my caches did NOT resolve the issue. On August 11, 2017 at 09:24:22, zeo...@gmail.com (zeo...@gmail.com) wrote: Master is failing with that right now. https://travis-ci.org/apache/metron#L2637 Jon On Fri, Aug 11, 2017 at 9:21 AM Otto Fowler <ottobackwa...@gmail.com> wrote:

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Otto Fowler
les/node-sass/lib/index.js:294:32) > > [ERROR] npm ERR! Darwin 15.5.0 > [ERROR] npm ERR! argv > "/Users/cstella/Documents/workspace/metron/fork/incubator-metron/metron- > interface/metron-alerts/node/node" > "/Users/cstella/Documents/workspace/metron/fork/incubator-m

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Otto Fowler
Can you create a jira for moving to yarn please. On August 11, 2017 at 10:17:40, Otto Fowler (ottobackwa...@gmail.com) wrote: Are there any other NPM modules where we are exposed to this risk? On August 11, 2017 at 10:11:22, RaghuMitra Kandikonda ( raghumitra@gmail.com) wrote: Ok

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Otto Fowler
PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Where are the bootstrap files? > > the @import ~bootstrap/….. looks like it is failing? > > > > On August 11, 2017 at 09:51:03, RaghuMitra Kandikonda ( > raghumitra@gmail.com) wrote: > > I am running a clean

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
in a JVM. This makes it very simple to reuse the same implementation (Stellar functions) across multiple contexts; the REPL, Streaming, and in a REST API. On Sun, Jul 2, 2017 at 10:06 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Bump > > > On June 13, 2017 at 00:11:52, Otto

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
misunderstood. My bad. On Fri, Jul 7, 2017 at 2:47 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > This was my original inclination and Casey’s as well when we spoke. I > think Nick has some good points however, so I created this thread. > > Thanks for replying! > > > On Jul

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
Wait, Matt are you saying the answer is ‘neither’ and we should have a 3rd layer? That both call? On July 7, 2017 at 14:47:06, Otto Fowler (ottobackwa...@gmail.com) wrote: This was my original inclination and Casey’s as well when we spoke. I think Nick has some good points however, so I

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Otto Fowler
are going to come out > of this discussion? > > I'm all for adding as many rest endpoints as possible. It makes our > platform much easier to understand and use for people who are not experts > on Metron internals. > > > On Jul 7, 2017, at 11:07 AM, Otto Fowler <ottobackwa...@gmail

Stellar: Is there a reason we don’t use ConvertUtils?

2017-07-07 Thread Otto Fowler
Is there a reason that we don’t use the ConversionUtils conversion function in stellar, as opposed to casting and checking null everywhere? This is wrt the args.

Re: [DISCUSS] Relocate Docker

2017-07-13 Thread Otto Fowler
I think that if it is not a ‘build breaking dependency’ it should be in a /contrib area and not in deployment. On July 13, 2017 at 12:05:00, Nick Allen (n...@nickallen.org) wrote: Having metron-docker at the top-level of the project seems to catch the attention of new users. Some then start

[DISCUSSION] METRON-1046 -> Stellar Files for multiple statement execution

2017-07-14 Thread Otto Fowler
https://issues.apache.org/jira/browse/METRON-1046 I was thinking this morning that managing stellar statements in the config json could become, and maybe is kind of unwieldy. To that end, if in say a parser configuration I can refer to a ‘file’ in zookeeper as an alternative, we would add the

Re: Elasticsearch 5.x upgrade

2017-07-17 Thread Otto Fowler
The plan around the mpack with regards to elastic search and solr support and options is extremely important and as of yet undecided. On July 17, 2017 at 03:00:42, Simon Elliston Ball ( si...@simonellistonball.com) wrote: > I assume you're taking about METRON-939. The upgrade is not a huge

Re: Post-parsing and Enrichment test framework

2017-07-08 Thread Otto Fowler
Hi Ali, Would you consider creating a jira with some test framework requirement ideas based on your experience using GetProfileTest? On July 8, 2017 at 07:11:53, Ali Nazemian (alinazem...@gmail.com) wrote: Hi Nick, Something like GetProfileTest is exactly what I am looking for. Although

Re: [DISCUSSION] METRON-1046 -> Stellar Files for multiple statement execution

2017-07-14 Thread Otto Fowler
nter model, can we use JSON Pointers? Will they work with Curator? Both of those get into some fairly obscure features, that would need to be studied. It also actually relates to the syntax question presented above. On 7/14/17, 6:17 AM, "Otto Fowler" <ottobackwa...@gmail.com> wrote:

Re: threatintel_taxii_load.sh throws exception

2017-07-10 Thread Otto Fowler
After upgrading to 0.4.0? I am not sure you should be running the threatintell_taxii_load.sh out of /usr/metron/0.3.1. What happens when you run it out of /usr/metron/0.4.1 ? On July 10, 2017 at 08:19:18, Vladimir Shlyakhtin ( vladimir.shlyakh...@sstech.us) wrote: Hello, After upgrading from

Re: ability to submit patches

2017-07-24 Thread Otto Fowler
Please reference : https://cwiki.apache.org/confluence/display/METRON/Metron+Development+Environment+Setup+Instructions For instructions on submitting a PR against a JIRA. Also, please note the PR checklist when submitting. On July 24, 2017 at 09:07:30, Artem Ervits (dbis...@gmail.com) wrote:

Re: [DISCUSS] Easing the ramp-up into contributing

2017-07-27 Thread Otto Fowler
Beyond this, I think we need to encourage PR review and testing as a way to contribute, along with documentation. So things that help with those topics will be good. Maybe an expended "ways to get involved including”, and why they are important. Increased participation leading to PR’s that don’t

Re: [GitHub] metron issue #673: METRON-1069: Ambari MPack documentation around developmen...

2017-07-27 Thread Otto Fowler
This is indeed awesome. What is also missing from my person experience and pov. is how to add a new rpm or extend the functionality. If you don’t add the rpm to the configuration, it won’t get deployed. How do you call Zookeeper? What get’s called when in the scripts? What where the steps

Re: Question about Metron-Docker

2017-06-28 Thread Otto Fowler
for a while. -Kyle > On Jun 28, 2017, at 11:10 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Has anyone run it off master? > Is anyone using it? > >

Re: Build failures

2017-06-28 Thread Otto Fowler
Also, if you are following an article or some documentation, can you tells what those are? On June 28, 2017 at 16:43:33, Ryan Merriman (merrim...@gmail.com) wrote: Can you confirm you're on the master branch? I see "metron-streaming" in your path to RestTestingUtil and that was changed a LONG

Re: [VOTE] Apache Metron 0.4.0 release

2017-07-03 Thread Otto Fowler
Great job everyone On July 4, 2017 at 00:49:59, Matt Foley (ma...@apache.org) wrote: The vote has passed, with: +1: 7 (3 binding, 4 non-binding) 0: none -1: none Thanks to all who tested and voted! I am now going through the release process and will “ANNOUNCE” when done. Thanks, --Matt On

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-02 Thread Otto Fowler
Bump On June 13, 2017 at 00:11:52, Otto Fowler (ottobackwa...@gmail.com) wrote: I have opened METRON–994 <https://issues.apache.org/jira/browse/METRON-994> : STELLAR Shell and management should front the METRON REST api As Stellar management functions start overlapping with the REST a

Re: [GitHub] metron issue #616: METRON-877 Extract core implementation and UDF support, c...

2017-07-03 Thread Otto Fowler
This still has not hit. Should we be concerned about merges/commits before it does? On July 2, 2017 at 19:27:43, mattf-horton (g...@git.apache.org) wrote: Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/616 This has been committed to

Re: Runbook is ready for review

2017-07-05 Thread Otto Fowler
Metron has graduated, and as such any reference to Apache Metron (Incubating) should be just Apache Metron. On July 5, 2017 at 09:48:37, Rita McKissick (rmckiss...@hortonworks.com) wrote: There are still a couple sections that are work in progress: * Enriching Threat Intelligence * Configuring

Re: [Request for Consensus Approval] dev branch for Stellar additional work

2017-07-05 Thread Otto Fowler
Yeah, this is part of why we need the guide On July 5, 2017 at 09:23:03, zeo...@gmail.com (zeo...@gmail.com) wrote: That all sounds pretty reasonable to me. My biggest concern would be attribution during step 5 - we would need to make sure it isn't squash merged like we typically do (assuming

Re: [Request for Consensus Approval] dev branch for Stellar additional work

2017-07-05 Thread Otto Fowler
Just want to say, we will need updated versions of the commuter’s guide for doing commits and reviews to branches like this. The current scripts we use will need to be updated ( as we go ), the we should at least have a plan to provide these guides before the first pr. On July 5, 2017 at

Re: [VOTE] Apache Metron 0.4.0 release

2017-06-27 Thread Otto Fowler
+1 * verified keys * ran : mvn -q -T 2C -DskipTests install && \ ] 6:04 PM mvn -q -T 2C org.jacoco:jacoco-maven-plugin:prepare-agent surefire:test@unit-tests && mvn -q \ org.jacoco:jacoco-maven-plugin:prepare-agent surefire:test@integration-tests && mvn -q \

Re: [ANNOUNCE] Apache Metron 0.4.0 release

2017-07-05 Thread Otto Fowler
Thank you Matt, your first metron release as well! Congratulations to the community. On July 5, 2017 at 16:35:47, Matt Foley (ma...@apache.org) wrote: Friends and Colleagues, I’m happy to announce the completion and release of Apache Metron 0.4.0. Besides a bunch of great new features, this is

Re: auto-install on bare metal

2017-04-25 Thread Otto Fowler
I failed at this today, but maybe it was the way I tried. An example would be great. On April 25, 2017 at 20:11:26, David Lyle (dlyle65...@gmail.com) wrote: Hi Dima, The same Ansible playbooks that work for EC2 and Vagrant will work for bare metal installations. The only difference is that

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
So, the copy resources configuration is still there, and not governed by the profile. Is it possible that since they both run in package, that they are out of order? On April 25, 2017 at 22:31:27, Otto Fowler (ottobackwa...@gmail.com) wrote: Having said that, i see the problem. I am very sorry

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
http://stackoverflow.com/questions/22150209/maven-changes-the-order-of-plugins-of-different-profiles https://issues.apache.org/jira/browse/MNG-2258 On April 25, 2017 at 22:33:58, Otto Fowler (ottobackwa...@gmail.com) wrote: So, the copy resources configuration is still there, and not governed

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
Having said that, i see the problem. I am very sorry How do I go about resubmitting if I can fix it? On April 25, 2017 at 22:29:56, Otto Fowler (ottobackwa...@gmail.com) wrote: OK, I’ll take a look. Sorry - I thought they read out of the RPMS dir and verified that they were produced both

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
Can you describe what ES issues you are working on? Fixing the repos got everything installed, but my ES components don’t stay running. I need to harvest the errors. On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwa...@gmail.com) wrote: Nm. sorry. I fixed it. On April 25, 2017 at 16

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-26 Thread Otto Fowler
in the outbox Casey? > > Jon > > On Wed, Apr 26, 2017, 6:53 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > > > What other thread? > > > > > > On April 25, 2017 at 19:56:56, Casey Stella (ceste...@gmail.com) wrote: > > > > Ok, I spun up that disc

Re: Normalization topology or separate normalization bolt for parsing topology

2017-04-26 Thread Otto Fowler
I’ve added this to the jira On April 26, 2017 at 09:28:54, Otto Fowler (ottobackwa...@gmail.com) wrote: What if you could implement your cleaning in Stellar functions, which would be in libraries that were loaded as plugins and available to all your parsers? my_field = ALI_CLEANMYFIELD

Re: auto-install on bare metal

2017-04-26 Thread Otto Fowler
do that. I created this quite a while ago, but it shows you the structure and how you could manage multiple environments with this method. https://github.com/nickwallen/metron-environments On Tue, Apr 25, 2017 at 9:46 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > I failed at t

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
lt;http://airmail.calendar/2017-04-25%2022:14:58%20EDT> ccs-fx-met-poc-86.dev.industrialdefender.com systemd[1]: elasticsearch.service failed. INFO 2017-04-26 08:55:19,962 <http://airmail.calendar/2017-04-26%2008:55:19%20EDT> Contr On April 26, 2017 at 08:27:27, Otto Fowler (ottobackwa...@gmail.com) wrote: C

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
build: http://imgur.com/yjkltES Regards, Anand On 4/26/17, 5:57 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: >Can you describe what ES issues you are working on? >Fixing the repos got everything installed, but my ES components don’t stay >running. >

Re: Normalization topology or separate normalization bolt for parsing topology

2017-04-26 Thread Otto Fowler
Hi, Are you doing this cleansing all in the parser or are you using any Stellar to do it? Can you create a jira? On April 26, 2017 at 08:59:16, Ali Nazemian (alinazem...@gmail.com) wrote: Hi all, We are facing certain use cases in Metron production that happen to be related to noisy stream.

Re: auto-install on bare metal

2017-04-26 Thread Otto Fowler
otally misunderstood what you're looking for. > Sorry. > > > > On Wed, Apr 26, 2017 at 9:59 AM, Otto Fowler <ottobackwa...@gmail.com> > > wrote: > > > >> Right, I think this : https://cwiki.apache.org/ > confluence/pages/viewpage. > >> action?pageId=6514

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
, 2017 at 7:02 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > File > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line > 293, in _call > > raise ExecutionFailed(err_msg, code, out, err) > > ExecutionFailed: Execution of 'ser

Re: auto-install on bare metal

2017-04-26 Thread Otto Fowler
26/2017 07:54 PM, Nick Allen wrote: Ok, then I must have totally misunderstood what you're looking for. Sorry. On Wed, Apr 26, 2017 at 9:59 AM, Otto Fowler <ottobackwa...@gmail.com ><mailto:ottobackwa...@gmail.com> wrote: Right, I think this : https://cwiki.apache.org/ co

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
Can I ask Dima, as comfortable you are in describing, what areas do you make changes or additions to that require you to maintain a fork? Custom parsers? Enrichments? Other? I have done some work, and have more planned and lined up to try to eliminate the requirement to develop in the metron

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
them to the decent level currently). We also merged our custom tools that extends Metron batching processing and store data in HDFS and Hive to be used by tableau, zeppelin, etc. Is that answers your question? Let me know if you want to know more. - Dima On 04/26/2017 08:29 PM, Otto Fowler wrote

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
Getting the same error in ES as before from what I can see On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwa...@gmail.com) wrote: I still can’t keep ES running, it dies. Also, I’m getting exceptions from enrichment trying to restart when the topology wasn’t running. Is there some

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
So the elasticsearch service is disabled. That is the issue. When ES calls get status ( service elasticsearch status ) it fails. I don’t know systemd too well, but does that mean we didn’t do the equiv of chkconfig on? On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwa...@gmail.com) wrote

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
Actually never mind. I am not sure how to read the exception. Is it failing to read the configuration? On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwa...@gmail.com) wrote: So the elasticsearch service is disabled. That is the issue. When ES calls get status ( service elasticsearch

Re: Quick Dev - Atlas Images

2017-04-26 Thread Otto Fowler
s? - How does the "time to start" compare for a "canned" release image as Dave suggested? On Tue, Apr 25, 2017 at 4:09 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > If it is docker, then it is just the Dockerfiles. > I think you can have vagrant use

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
that passes data through parsers/enrichments classes and stores in Hive. So what you suggesting is to have internal repo with all our changes alone and to merge it with Apache Metron whenever we need to get full build? - Dima On 04/26/2017 09:09 PM, Otto Fowler wrote: Thanks! Parsers

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
...@gmail.com (zeo...@gmail.com) wrote: I'm also interested to know why that's important at such a small scale. Jon On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > I am following > > https://community.hortonworks.com/articles/60805/deploying-a-fresh

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
. On Wed, Apr 26, 2017 at 4:48 PM, Nick Allen <n...@nickallen.org> wrote: > Yes, Otto. I just experienced that myself. It is a bug that we should > create a JIRA for. > > I was able to work around it by just using "start" instead of "restart". > >

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
file. network.host: ["_lo:ipv4_","_eth0:ipv4_"] On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > I think it is literally crashing trying to load the config file > > > > On April 26, 2017 at 15:00:25, Michael Miklavc

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
So now I’m getting errors for ETH0 I think that is one of the things that has to change on centos7? Can you check what you have? On April 26, 2017 at 15:20:18, Nick Allen (n...@nickallen.org) wrote: Yes, I am also running on CentOS 7. On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler <ottobac

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
would file a Jira. On Wed, Apr 26, 2017 at 1:41 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Ok, I did ifconfig, i mean ip a and got the right values > that and the brackets and it looks like es is up. > > Now my next problem is prob. a bug. > > Ambari thinks it

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Do we have any async methods in the current api I could look at? On April 27, 2017 at 10:45:09, Otto Fowler (ottobackwa...@gmail.com) wrote: I thought as much. I have not worked with the framework before however. On April 27, 2017 at 10:20:20, Ryan Merriman (merrim...@gmail.com) wrote: I

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
-> configurations to ‘available area’ ( ES, ParserConfig, ) UI -> Create Parser Instance -> take available, clone, edit save/deploy for XXX On April 27, 2017 at 10:28:18, Otto Fowler (ottobackwa...@gmail.com) wrote: Maybe the templates can be installed somewhere, not ES but some store. The UI

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Also, if you want to help that would be great ;) On April 27, 2017 at 09:30:59, Otto Fowler (ottobackwa...@gmail.com) wrote: So, assuming ( I know I know ) that METRON-777 eventually lands, it will have lain the framework for the extension system for parsers out, including the capability

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
to think about is whether this belongs in the metron-rest package (personally i think it probably does) or whether it is a separate micro-service. Anyway, happy to help in any way I can on this. Simon > On 27 Apr 2017, at 15:09, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Al

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Maybe the templates can be installed somewhere, not ES but some store. The UI can list the available templates, and you can edit and deploy from the UI? They can still be packaged with the parser, just not go to ES. Does that address your concerns? On April 27, 2017 at 10:25:37, Otto Fowler

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
(HDFS read/write, Zookeeper read/write). Asynchronous functions are also supported so no issues there. On Thu, Apr 27, 2017 at 9:09 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Also, if you want to help that would be great ;) > > > On April 27, 2017 at 09:30:59, Otto

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
that was not running. The question is - is the state wrong _and_ the exception catch or not. I am guessing I should start a Jira for that one? On April 26, 2017 at 15:31:05, Otto Fowler (ottobackwa...@gmail.com) wrote: So now I’m getting errors for ETH0 I think that is one of the things that has

[DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
So, assuming ( I know I know ) that METRON-777 eventually lands, it will have lain the framework for the extension system for parsers out, including the capability to create parsers outside the metron tree. The next step is METRON-258, side loading of parsers. This will be the effort to

Re: Ansible Docker fails to build latest

2017-04-21 Thread Otto Fowler
OK, the issue is the old old maven that is in the ansible image. Changing it to 3.3.9 resolved. On April 21, 2017 at 17:21:02, Otto Fowler (ottobackwa...@gmail.com) wrote: Yes. I am following the readme in there and starting the image, then just running mvn package -DskipTests On April 21

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
17-04-25 14:13:03,429 <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping installation of existing package curl 2017-04-25 14:13:03,430 <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5

Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Hi, Should I see the repo tab in the ambari wizard for metron at this point? Asking for a friend who can’t get the wizard to work :)

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
PR probably would have done so. Can you elaborate on "prior capability wrt centos 6.x work again" a bit? On Apr 25, 2017 10:59 AM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: > Also, I can’t find any list discussion of the issue or other approaches. > > &g

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
Also, I can’t find any list discussion of the issue or other approaches. On April 25, 2017 at 12:48:46, Otto Fowler (ottobackwa...@gmail.com) wrote: What I am looking for is the prior capability wrt centos 6.x work again ( which I think my pr fixing the docker in docker issue would do

Re: Silly question about zk_load_configs.sh

2017-04-24 Thread Otto Fowler
So, this script is in common, but: export JAR=metron-parsers-$METRON_VERSION-uber.jar Is the jar it executes. The class it references is : export CLASSNAME=“org.apache.metron.common.cli.ConfigurationManager" So, why is it calling the parsers jar?

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
, or alternatively create some Ansible roles to make the REST calls that Ambari requires to stand up a cluster. -D... [1] https://cwiki.apache.org/confluence/display/AMBARI/Blueprints On Tue, Apr 25, 2017 at 11:35 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Let m

Re: metron_full_install playbook with small_cluster and multiple hosts

2017-04-25 Thread Otto Fowler
deployments, I only saw one build being executed. I don't know what you mean by "set to become", Become is set to false because I didn't want to sudo to execute the build. -D... On Tue, Apr 25, 2017 at 10:51 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Now that I

Re: Quick Dev - Atlas Images

2017-04-25 Thread Otto Fowler
If it is docker, then it is just the Dockerfiles. I think you can have vagrant use docker as a back end too right? On April 25, 2017 at 14:34:14, Nick Allen (n...@nickallen.org) wrote: >> I hadn't really reasoned about the notion of a "released" Quick Dev image, but I can see a lot of value in

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Nm. sorry. I fixed it. On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwa...@gmail.com) wrote: Ok, now I see the repos in the ‘pick version’ screen, but it is erring on the f://localrepo even though the folder exists, there is no repodata/repomd.xml. What is the command to create

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
kaging/ambari/metron-mpack/src/main/ > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > > stderr: > > > > Traceback (most recent call last): > >

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
, Apr 25, 2017 at 3:44 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > 2.4.1 as a matter of fact > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65...@gmail.com) wrote: > > Any chance they're running Ambari < 2.4.2? > > -D... > > > On Tue,

METRON-777 and rest interface for installing parsers

2017-04-28 Thread Otto Fowler
I have enough of a rest interface around installing parsers that I’m thinking of merging it down into 777 and get them reviewed together, as apposed to following on. I know there will be a bunch of rework from suggestions coming, it would be easier to refactor on the same branch. Thoughts? Is 777

Re: Silly question about zk_load_configs.sh

2017-04-24 Thread Otto Fowler
24, 2017 at 9:54 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > too bad there isn’t a pr out for that kind of stuff ;) > > There are no stellar functions in parsers, except what is brought in from > common, why doesn’t this just call common? > You don’t need parsers jar f

Status of METRON-153

2017-04-25 Thread Otto Fowler
https://issues.apache.org/jira/browse/METRON-153 What is the other approach that is mentioned here? Was it implemented? The ansible build still does not support this and I was thinking of looking at it.

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
in my Metron cluster, I must run on RHEL 7. I assume that will be common across many enterprises. Semi-recentlyI took a stab at CentOS 7 support but it was a bit of a rough go and I dropped it down on my priority list. Jon On Tue, Apr 25, 2017 at 11:28 AM Otto Fowler <ottobackwa...@gmail.com>

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-08-08 Thread Otto Fowler
: I’d be happy to collaborate in specifying and implementing this solution, if you guys like. The stuff currently in Stellar is a good starting point, so it really wouldn’t be a big job. --Matt On 7/7/17, 12:01 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: No, you are ri

Re: Profiler statistics NaN

2017-08-09 Thread Otto Fowler
(ceste...@gmail.com) wrote: Well, we need that too :) What're you thinking, procedures for stellar? On Wed, Aug 9, 2017 at 4:42 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > What we need, is a way to package up some ‘recipes’ for stellar. > If many people are going to do th

Re: Profiler statistics NaN

2017-08-09 Thread Otto Fowler
What we need, is a way to package up some ‘recipes’ for stellar. If many people are going to do this operation, then a more friendly set of facade functions, or some thing would work. On August 9, 2017 at 16:38:48, Casey Stella (ceste...@gmail.com) wrote: Yeah, I'm leaning toward STATS_ADD or

[DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Otto Fowler
A feature branch As discussed in the community meeting we are going to create a ‘feature’ branch for METRON–777 and it’s associated PR’s. The purpose behind the feature branch is to more formally stage a large PR or set of PR’s to allow for for better review participation ( of the pr and changes

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Otto Fowler
WRT : regression fixes, I would also like us to consider putting these the initial 777 to feature branch PR as an option. On August 23, 2017 at 09:56:33, Otto Fowler (ottobackwa...@gmail.com) wrote: A feature branch As discussed in the community meeting we are going to create a ‘feature

REST and ManagementUI : Grok

2017-08-17 Thread Otto Fowler
Question, How does the UI decide, when you edit a parser, to show the grok pattern field? I have branch where I have broken that, for example if I add a new configuration, I see the grok pattern field, until I actually select GROK as the parser type, then it goes away. Also, just to confirm,

Re: REST and ManagementUI : Grok

2017-08-17 Thread Otto Fowler
-parser-config/sensor-parser-config.component.ts. That's where the check is done to determine if a grok parser has been selected. I would also check the browser console for errors. On Thu, Aug 17, 2017 at 10:51 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > That is the “I have a branc

Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
Anyone know what this is? [ERROR] No plugin found for prefix 'jacoco' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/home/travis/.m2/repository), central ( https://repo.maven.apache.org/maven2)] -> [Help 1]

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
Just for anyone else https://docs.travis-ci.com/user/caching/#Clearing-Caches On May 3, 2017 at 11:45:26, Otto Fowler (ottobackwa...@gmail.com) wrote: > I am not sure I know how to do that in Travis. I will look when I get back > > On May 3, 2017 at 11:35:34, Ryan Merriman (merrim...@

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
n Wed, May 3, 2017 at 10:26 AM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > https://travis-ci.org/ottobackwards/incubator-metron/builds/228364894?utm_ > source=email_medium=notification > > On May 3, 2017 at 11:12:15, Justin Leet (justinjl...@gmail.com) wrote: &

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Otto Fowler
elasticsearch/defaults/main.yml#L19> - 'network.host: ["_{{ elasticsearch_network_interface }}:ipv4_","_local:ipv4_"] < https://github.com/apache/incubator-metron/blob/Metron_0.3.1/metron-deployment/roles/elasticsearch/tasks/elasticsearch.yml#L69> On Wed, May 3

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Otto Fowler
portMasterNodeAction.java#L148 > which suggests that a cluster action has been requested on a local > (loopback) address. This is not > surprising given what I’ve learned about the semantics of network.host > with wildcard address. > See next message, item C. Basically, while the wi

  1   2   3   4   5   6   >