Re: [DISCUSS] Alerts UI: Loading state while fetching data

2019-07-24 Thread Nick Allen
Yes! I think this is sorely needed. Would this also include indicating when an error has occurred in the backend call? That might also be helpful and somewhat related to METRON-2190. On Wed, Jul 24, 2019 at 9:27 AM Tibor Meller wrote: > Hi all, > > I think it would great to have a loading sta

[DISCUSS] Deprecate Least Recently Used Pruner

2019-08-13 Thread Nick Allen
As part of https://github.com/apache/metron/pull/1470, I found it difficult to update the "Least Recently Used Pruner" to work with HBase 2.0.2. I am sure that given more time and effort, I could make it work, but is it worth it? This is a feature that I myself am not familiar with. I do not know

Re: [DISCUSS] Deprecate Least Recently Used Pruner

2019-08-13 Thread Nick Allen
PM Otto Fowler wrote: > Can you summarize what it does? Is it from OpenSOC? > > > > > On August 13, 2019 at 17:53:52, Nick Allen (n...@nickallen.org) wrote: > > As part of https://github.com/apache/metron/pull/1470, I found it > difficult > to update the "Least

Re: [DISCUSS] Curator client upgrade

2019-09-17 Thread Nick Allen
+1 to making the change on the feature branch. We don't really know how this might affect master which is still building against HDP 2.6, nor is it strictly needed there. Going to Curator 4.0.0 is only needed due to the HDP 3.1 upgrade. This is also likely to get more focused testing cycles in

Re: [DISCUSS] deprecate misleading install methods and docs?

2019-10-29 Thread Nick Allen
+1 On the remove option. I think we should *completely remove* the automated AWS deployment mechanism because it has been too difficult to maintain, deploys an unsecure cluster by default, and is not the preferred installation path for AWS. If a user wants to deploy to AWS, they should launch th

Branch Cleanup

2019-10-29 Thread Nick Allen
Heads up... I accidentally pushed a feature branch to Apache called METRON-2223. This was my mistake. I have just deleted the branch. My apologies

Re: [DISCUSS] deprecate misleading install methods and docs?

2019-11-19 Thread Nick Allen
FYI - See the following which removes the automated AWS deployment mechanism. https://issues.apache.org/jira/browse/METRON-2321 https://github.com/apache/metron/pull/1565 On Tue, Oct 29, 2019 at 10:23 AM Nick Allen wrote: > +1 On the remove option. I think we should *completely remove*

[DISCUSS] Next Release - Life After 0.7.1

2019-12-05 Thread Nick Allen
Hello Metron'ers - I would like to make the case that it is time for us to cut the next Apache Metron release. - Our last release was 0.7.1 on May 15th . It has b

Re: JUnit 5 PR merged into master

2019-12-07 Thread Nick Allen
Thanks for the hard work on that upgrade and this very useful highlight reel. On Sat, Dec 7, 2019, 10:17 AM Justin Leet wrote: > Hi all, > > The JUnit 5 migration PR has been merged to master. From this point > forward, please use the newer interfaces and methods. There are plenty of > examples

Re: [DISCUSS] Next Release - Life After 0.7.1

2019-12-13 Thread Nick Allen
really solid > >> release. > >> > > > >> > > https://github.com/apache/metron/pull/1568 should be in before > >> release. > >> > It > >> > > addresses an issue with our validation of dependencies_with_url.csv > >>

Re: Possible approach to solve GeoIP update?

2020-04-15 Thread Nick Allen
That seems like a viable solution to me. On Thu, Apr 9, 2020 at 7:58 PM Yerex, Tom wrote: > Good afternoon, > > Reviewing hxxps:// > issues.apache.org/jira/projects/METRON/issues/METRON-2340 I'm attempting > to sketch out a rough solution and I would like guidance from more > experienced minds

Re: Centos6 and Centos7 instructions

2020-04-15 Thread Nick Allen
Hi Tom - The source for https://metron.apache.org/current-book/metron-deployment/development/centos6/index.html is contained in the README at `metron/metron-deployment/development/centos6/README.md`. When a release happens we have a script that generates the site book from the various documentati

Re: Development Activity has dropped to effectively 0, what should we do?

2020-04-17 Thread Nick Allen
This is a good discussion and one that I haven't fully grappled with in my own mind yet. I'll have more to add, but I just want to chime in on the topic of Ambari at this point. ### Ambari and the Paywall The problem with Ambari is that its installation mechanism requires a repository of compiled

Re: Development Activity has dropped to effectively 0, what should we do?

2020-04-21 Thread Nick Allen
t; docker image that is designed to connect with other dockerized applications > such as Storm, Kafka, etc..? > > --Tom. > > On 2020-04-17, 11:27 AM, "Nick Allen" wrote: > > This is a good discussion and one that I haven't fully grappled with > in my >

Re: [VOTE] Move Apache Metron to the Apache Attic and Dissolve PMC

2020-11-16 Thread Nick Allen
+1 On Mon, Nov 16, 2020 at 11:41 AM zeo...@gmail.com wrote: > +1 > > -- > Jon Zeolla > @jonzeolla > > PittSec | BSidesPGH | SteelCityInfoSec > > On Mon, Nov 16, 2020, 11:33 AM Casey Stella wrote: > > > +1 > > > > On Mon, Nov 16, 2020 at 09:01 Justin Leet wrote: > > > > > Hi all, > > > > > > Th

Failure to Deploy "Quick Dev"

2017-04-21 Thread Nick Allen
Not sure if I am doing something stupid, but I cannot deploy "Quick Dev" from master currently. Full details in https://issues.apache.org/jira/browse/METRON-872.

Re: Reducing Warnings in Build

2017-04-21 Thread Nick Allen
Per (2), I think it makes sense to make the charset configurable, but with the proposal of 3 separate settings, wouldn't things blow up horribly if the Parsers are producing UTF-8, but Enrichment is expecting UTF-16? They are not even speaking the same language, no? This makes me think that we ne

Quick Dev - Atlas Images

2017-04-24 Thread Nick Allen
Right now, we have the images that get pushed to Atlas for Quick Dev versioned independently from the rest of Metron. We currently have versions 0.1.0 and 0.2.0. What happens when a user downloads an official release of Metron, like 0.3.1, and

Re: Quick Dev - Atlas Images

2017-04-25 Thread Nick Allen
should release it as a convenience binary signed and hosted alongside the > other release artifacts. Meantime, we could keep the incremental versions > of Quick Dev in Atlas. > > Anyway, I think it's a really interesting notion. > > -D... > > > On Mon, Apr 24, 2017 a

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Nick Allen
createrepo On Tue, Apr 25, 2017 at 4:42 PM, Otto Fowler wrote: > Ok, now I see the repos in the ‘pick version’ screen, but it is erring on > the f://localrepo > even though the folder exists, there is no repodata/repomd.xml. > > What is the command to create a local repo? > > > > On April 25, 20

Re: Normalization topology or separate normalization bolt for parsing topology

2017-04-26 Thread Nick Allen
> For some reason, the incoming data do not look like in the way that has to be. In my mind that would be something for your parser to handle. On Wed, Apr 26, 2017 at 9:43 AM, Ali Nazemian wrote: > Having Stellar function for the normalization is very cool actually. > > Casey, how are you going

Re: auto-install on bare metal

2017-04-26 Thread Nick Allen
Here is an example of how you might do that. I created this quite a while ago, but it shows you the structure and how you could manage multiple environments with this method. https://github.com/nickwallen/metron-environments On Tue, Apr 25, 2017 at 9:46 PM, Otto Fowler wrote: > I failed at thi

Re: auto-install on bare metal

2017-04-26 Thread Nick Allen
ent changes to allow building in docker > again. > > > On April 26, 2017 at 09:54:26, Nick Allen (n...@nickallen.org) wrote: > > Here is an example of how you might do that. I created this quite a while > ago, but it shows you the structure and how you could manage multiple > en

Re: auto-install on bare metal

2017-04-26 Thread Nick Allen
a > > On 04/26/2017 07:54 PM, Nick Allen wrote: > > Ok, then I must have totally misunderstood what you're looking for. > Sorry. > > > > On Wed, Apr 26, 2017 at 9:59 AM, Otto Fowler > > wrote: > > > >> Right, I think this : https://cwiki.apache

Re: Quick Dev - Atlas Images

2017-04-26 Thread Nick Allen
rant use docker as a back end too right? > > > > On April 25, 2017 at 14:34:14, Nick Allen (n...@nickallen.org) wrote: > > >> I hadn't really reasoned about the notion of a "released" Quick Dev > image, > but I can see a lot of value in having a versione

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Nick Allen
Check on the `network_hosts` setting. I just ran into the same issue. I had to add brackets around it otherwise the file was not valid YAML and it would crash trying to load the config file. network.host: ["_lo:ipv4_","_eth0:ipv4_"] On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler wrote: > I thin

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Nick Allen
Yes, I am also running on CentOS 7. On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler wrote: > Would i have to change that on Centos 7? > > > On April 26, 2017 at 15:12:38, Nick Allen (n...@nickallen.org) wrote: > > Check on the `network_hosts` setting. I just ran into the same iss

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Nick Allen
hat is one of the things that has to change on centos7? > Can you check what you have? > > > > On April 26, 2017 at 15:20:18, Nick Allen (n...@nickallen.org) wrote: > > Yes, I am also running on CentOS 7. > > On Wed, Apr 26, 2017 at 3:19 PM, Otto Fowler > wrote: > &

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Nick Allen
I can create the JIRA. I capture the logs. On Wed, Apr 26, 2017 at 4:48 PM, Nick Allen wrote: > Yes, Otto. I just experienced that myself. It is a bug that we should > create a JIRA for. > > I was able to work around it by just using "start" instead of "restart&qu

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Nick Allen
calls. I wonder what the best practice is? > I can’t seem to find the ambari mpack programming documentation ;) > > > On April 26, 2017 at 16:50:18, Nick Allen (n...@nickallen.org) wrote: > > I can create the JIRA. I capture the logs. > > On Wed, Apr 26, 2017 at 4:48 PM, Ni

Re: [VOTE] Metron 0.4.0 release (RC2)

2017-04-28 Thread Nick Allen
-1 binding I agree with you Anand. It should work out-of-the-box. The fix is simple. On Fri, Apr 28, 2017 at 6:56 AM, Anand Subramanian < asubraman...@hortonworks.com> wrote: > -1 (non-binding) > > Validated on a 12-node openstack CentOS 7 setup. > > - Ran 'mvn clean package' [PASS] > - Able

[DISCUSS] Kerberos First

2017-05-01 Thread Nick Allen
I hate dealing with Kerberos. It is a pain to setup, it is a pain to work with, it has its own learning curve, *but it is absolutely necessary*. Due to the sensitive nature of Metron's use case, most of our users should be using Kerberos as part of a defense-in-depth strategy to protect sensitive

Re: [DISCUSS] Update Metron Release Documentation

2017-05-01 Thread Nick Allen
One major benefit of the site-book is that we can maintain docs for previous releases of Metron. Unless there is a major technical hurdle, I think we should do so. On Mon, May 1, 2017 at 10:06 AM, zeo...@gmail.com wrote: > Just bringing up this thread again, as we're going to have two books as o

Re: [DISCUSS] Kerberos First

2017-05-01 Thread Nick Allen
applies, > and it would still be good to allow QuickDev to come up without Kerberos as > a simplified environment when that’s desirable. > > > > Thanks, > > --Matt > > > > From: Nick Allen > Reply-To: "dev@metron.apache.org" > Date: Monday, May 1

Re: Normalization topology or separate normalization bolt for parsing topology

2017-05-02 Thread Nick Allen
Before worrying about how to ingest this 'noisy' data, I would want to better understand root cause. If you cannot even get a valid date format, are you sure the data can be trusted? Rather than bending over backwards to try to ingest it, I would first make sure the telemetry is not totally bogus

Re: Normalization topology or separate normalization bolt for parsing topology

2017-05-02 Thread Nick Allen
rtance of having a normalisation > step in Metron processing chain. > > I still think there is no guarantee to have an entirely clear and > well-defined message in the real world use case. If we recognise this > situation as a problem, then finding a high performance and flexible > s

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-02 Thread Nick Allen
I don't know how to uninstall, but you can reinstall by passing the --force flag On Tue, May 2, 2017 at 2:33 PM, Otto Fowler wrote: > Do you know how to uninstall an mpack from the cli? > > > On May 2, 2017 at 14:27:02, Otto Fowler (ottobackwa...@gmail.com) wrote: > > Are you saying that the de

Re: Normalization topology or separate normalization bolt for parsing topology

2017-05-03 Thread Nick Allen
y, it would be better that official Metron developers focus on Metron features instead of developing generic parsers. Thanks, Ali On Wed, May 3, 2017 at 3:03 AM, Nick Allen wrote: > Yes, and currently that normalization step is the Parsers. > > I am not saying the message has to be ent

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Nick Allen
It only worked "good enough" on Ansible because it was mainly used for deploying to a controlled environment where we know the interface names; aka Vagrant/Single Node. It did not work well at all on environments other than Vagrant/Single Node. The work that was done with Elasticsearch and Ambari

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Nick Allen
working out-of-the-box here, Matt? On Wed, May 3, 2017 at 8:30 AM, Nick Allen wrote: > It only worked "good enough" on Ansible because it was mainly used for > deploying to a controlled environment where we know the interface names; > aka Vagrant/Single Node. > > It di

Re: Failure to Deploy "Quick Dev"

2017-05-03 Thread Nick Allen
; >> From lira: >> >> I 'think' that quickdev is actually build from full_dev, with metron >> installed already. So it may be that we need a new image built to make >> this >> not an upgrade situation? >> >> >> On April 21, 2017 at 10:37:3

Re: MaaS + Apache Twill ?

2017-05-03 Thread Nick Allen
I think had we known that Twill existed a few years ago we would > have considered it, but given where we are today I don't see a reason to > switch. > > Thanks, > James > > 18.04.2017, 12:30, "Nick Allen" : > >> Can Twill handle long-running applicati

Re: [GitHub] incubator-metron issue #562: METRON-915 add node and npm to platform_info.sh

2017-05-03 Thread Nick Allen
I think it still makes sense to add these to the platform script. What if the mvn plugin breaks or starts working in an unexpected way? This would help us uncover issues there. But good point to bring up, Ryan. On Wed, May 3, 2017 at 8:32 AM, Ryan Merriman wrote: > We are using a mvn plugin t

Re: Failure to Deploy "Quick Dev"

2017-05-03 Thread Nick Allen
nickwallen Thanks. On Wed, May 3, 2017 at 9:03 AM, David Lyle wrote: > Hi Nick, > > You do. just need to set up an Atlas account and shoot over the name. > > -D... > > > On Wed, May 3, 2017 at 8:44 AM, Nick Allen wrote: > > > Who has the credentials to be a

Re: [DISCUSS] platform_info.sh and PR template / Dev Guidelines

2017-05-03 Thread Nick Allen
+0 I am not a fan of the PR template. In my opinion, there is too much text already and I ended up just skipping over it. I think it gets in the way of a contributor actually describing their change. Just my opinion. I know many others disagree and find value in it. On Wed, May 3, 2017 at 8:2

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Nick Allen
ing with small_cluster / ansible was that it just > > worked at the time to > > my centos 6.9 esxi cluster. > > > > > > On May 3, 2017 at 08:30:59, Nick Allen (n...@nickallen.org) wrote: > > > > It only worked "good enough&quo

Re: [DISCUSS] REST + ambari

2017-05-08 Thread Nick Allen
As opposed to using the Ambari REST API to get this information? On Mon, May 8, 2017 at 8:06 AM, Otto Fowler wrote: > I was thinking about have an ambari ‘service’ in the rest api. > The initial purpose would be to be able to retrieve ambari configuration > variables for the metron service and

Re: [DISCUSS] Code Style

2017-05-08 Thread Nick Allen
+1 Good points, Justin. I am onboard. On Mon, May 8, 2017 at 9:29 AM, Justin Leet wrote: > I've been taking a look at setting up checkstyle per > https://issues.apache.org/jira/browse/METRON-746. > > Given that we don't actually enforce any style right now (saying we use > Sun's is not the sa

Re: Why bro parser allows periods in keys?

2017-05-09 Thread Nick Allen
​The ES indexer replaces periods, I believe. Are you seeing periods hit Elasticsearch?​ And architecturally that kind of logic should be done in the indexers anyways. On Tue, May 9, 2017 at 9:41 AM, zeo...@gmail.com wrote: > Is there a reason why the bro parser allows periods >

Re: [DISCUSS] Enrichment Split/Join issues

2017-05-16 Thread Nick Allen
I would like to see us just migrate wholly to Stellar enrichments and remove the separate HBase and Geo enrichment bolts from the Enrichment topology. Stellar provides a user with much greater flexibility than the existing HBase and Geo enrichment bolts. A side effect of this would be to greatly

Re: [DISCUSS] Enrichment Split/Join issues

2017-05-16 Thread Nick Allen
uld be to > have just stellar enrichments, though. You can do every one of the other > enrichments in Stellar and it would greatly simplify that config above. > > > > On Tue, May 16, 2017 at 11:59 AM, Nick Allen wrote: > > > I would like to see us just migrate wholly to Stel

Re: [DISCUSS] Enrichment Split/Join issues

2017-05-16 Thread Nick Allen
hing is missing, and could drive > things like replay of the message to retrospectively enrich when things > have calmed down. > > Simon > > > On 16 May 2017, at 17:25, Nick Allen wrote: > > > > Ah, yes. Makes sense and I can see the value in the parallelism tha

Re: Trying to spin up Metron in EC2: Failed

2017-05-19 Thread Nick Allen
I am not sure what the issue is. Nothing is jumping out at me. Weird issues can arise when the machine that you are running the Ansible deployment from has intermittent connectivity issues to the AWS data center. Is Amazon's us-west-2 data center the one closest to you? You might try a differen

Re: Trying to spin up Metron in EC2: Failed

2017-05-19 Thread Nick Allen
Missed one important point. You actually choose the data center here [2] in the configuration file. [2] https://github.com/apache/metron/blob/master/metron-deployment/amazon-ec2/conf/defaults.yml#L43 On Fri, May 19, 2017 at 1:52 PM, Nick Allen wrote: > I am not sure what the issue

Re: [Discuss] Improving new developer onboarding experience

2017-05-19 Thread Nick Allen
Hi Marc - Welcome to the community. > Vagrant quick dev env does not boot for me. This relies on an image stored in Atlas. There was recently a commit that broke that image. We just need to update the image stored in Atlas. Until we do, feel free to use Full Dev. > The environment variables

Re: [Discuss] Improving new developer onboarding experience

2017-05-19 Thread Nick Allen
Also, there are usually people on #apache-metron on Freenode. If you are having trouble and need to ask a question, you can try there too. Jon and Otto tend to be our community leaders there. Many, many thanks to them for that. They are super helpful. I need to join more often. On Fri, May 19

Re: Trying to spin up Metron in EC2: Failed

2017-05-23 Thread Nick Allen
Good. Glad to hear that. On Tue, May 23, 2017 at 6:50 PM, Laurens Vets wrote: > Hmmm, deploying Metron in AWS from another EC2 instance seems to work > fine... > > I have a fully deployed 10 node Metron install now. > > On 2017-05-19 10:52, Nick Allen wrote: > >> I

Re: [Discuss] Cyber Security Asset Management for Metron

2017-05-24 Thread Nick Allen
I think the addition of a graph capability would be very powerful. I know many who would love the idea, but I know of no implementations that have occurred. It might be good to discuss in the community specific use cases that would be enabled by a graph database. That might help to flesh out the

Re: [INCOMING] Metron 0.4.0 release (RC3)

2017-06-01 Thread Nick Allen
Sounds good, Matt. Looking forward to cutting this release. On Thu, Jun 1, 2017 at 5:17 PM, Matt Foley wrote: > Hi all, > > Now that METRON-844 is in, I plan to proceed with the 0.4.0 release > candidate. I think 844 was the last item we considered a must-have for the > 0.4.0 release, and we w

Re: [Discussion] About the wiki….

2017-06-13 Thread Nick Allen
+1 for migrating away from the Wiki. I am not a fan. It is too difficult to keep in-sync with the source code. It also doesn't get the loving care and review that a PR does in our community. On Tue, Jun 13, 2017 at 2:27 PM, zeo...@gmail.com wrote: > I suggested in the past and got some buy in,

Profiler Deck

2017-06-19 Thread Nick Allen
I created a deck that walks through the current state of the Profiler. I thought this might be helpful to others on the list. Feel free to share any feedback or questions. https://www.slideshare.net/NickAllen4/apache-metron-profiler FYI - Viewing on Slideshare, corrupts a few images. If you d

Re: Metron Release rules

2017-06-26 Thread Nick Allen
+1 Yes, definitely cruft. Good find. On Mon, Jun 26, 2017 at 9:50 PM, Matt Foley wrote: > The Release Process document at https://cwiki.apache.org/ > confluence/display/METRON/Release+Process > currently states that a file named DISCLAIMER is a required artifact in > the release package. The

Re: [VOTE][PROPOSAL] minor changes to release process

2017-07-06 Thread Nick Allen
+1 I think that makes a lot of sense. On Wed, Jul 5, 2017 at 5:47 PM, Matt Foley wrote: > (The below proposal is also stated in https://issues.apache.org/ > jira/browse/METRON-1020 ) > > The following proposed changes are small, but not just editorial in > nature, hence will require vote of the

Re: Post-parsing and Enrichment test framework

2017-07-07 Thread Nick Allen
For experimenting or validating specific Stellar expressions, the Stellar Shell is perfect. To do this, you just have to remember than when your Stellar expressions execute all of the fields of the message are in-scope. For example, here is a quick session where I mock-up some logic that sends a

Re: Post-parsing and Enrichment test framework

2017-07-07 Thread Nick Allen
amples get you 80% there? On Fri, Jul 7, 2017 at 10:54 AM, Nick Allen wrote: > For experimenting or validating specific Stellar expressions, the Stellar > Shell is perfect. To do this, you just have to remember than when your > Stellar expressions execute all of the fields of the mes

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
Like you mentioned, Otto, I think it makes more sense to have a REST API that is backed by Stellar functions executed in a JVM. That is, the REST API simply executes the right Stellar functions in a JVM. This makes it very simple to reuse the same implementation (Stellar functions) across multipl

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
; of this discussion? > > I'm all for adding as many rest endpoints as possible. It makes our > platform much easier to understand and use for people who are not experts > on Metron internals. > > > On Jul 7, 2017, at 11:07 AM, Otto Fowler > wrote: > > > >

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
eason the issue came up is because there isn’t > already a Config API that is simple to call from the REST layer. It is > correct that the REST layer shouldn’t have to “fix” that, but neither > should it hack the solution by invoking Stellar. The correct architectural > plac

Re: [DISCUSS] METRON-994 -> Rest v. Stellar ( api of record )?

2017-07-07 Thread Nick Allen
ll from the REST layer. It is correct that > the REST layer shouldn’t have to “fix” that, but neither should it hack the > solution by invoking Stellar. The correct architectural place for a simple > Config API is Configuration. > > Thanks, > --Matt > > On 7/7/17, 10:01 AM, &quo

Metron REST - Logging Config

2017-07-11 Thread Nick Allen
How do I configure logging for Metron REST on a deployed host? Right now a log4j.properties file gets packaged into the metron-rest JAR itself. Is there is an easy way that I am missing?

[DISCUSS] Relocate Docker

2017-07-13 Thread Nick Allen
Having metron-docker at the top-level of the project seems to catch the attention of new users. Some then start using metron-docker to explore/try-out/demo Metron. The metron-docker code that we have is not well-suited for this purpose. It is only really useful for development. It is not regular

Re: [DISCUSS] Relocate Docker

2017-07-13 Thread Nick Allen
On Thu, Jul 13, 2017 at 12:30 PM Laurens Vets wrote: > > > On 2017-07-13 09:04, Nick Allen wrote: > > > Having metron-docker at the top-level of the project seems to catch the > > > attention of new users. Some then start using metron-docker to > > > explore/try

Re: [REQUEST] Contributor rights in Jira

2017-07-14 Thread Nick Allen
Hi Laurens - It seems that I do not have the access that I should in JIRA. I soon as I get my access worked out, I can take care of this for you. On Wed, Jul 12, 2017 at 5:21 PM, Laurens Vets wrote: > Hello, > > Could a PMC member please grant my Jira account contributor rights? I'd > like to st

Re: Metron REST - Logging Config

2017-07-14 Thread Nick Allen
n Fri, Jul 14, 2017 at 9:52 AM, Ryan Merriman wrote: > The only way I know of is to change log4j.properites. Did you every figure > out a better way? > > On Tue, Jul 11, 2017 at 2:10 PM, Nick Allen wrote: > > > How do I configure logging for Metron REST on a deployed

Re: Metron REST - Logging Config

2017-07-14 Thread Nick Allen
Actually per step 3, this is what the property should look-like. Note that "file:" has to precede the path. -Dlog4j.configuration=file:/path/to/log4j.properties On Fri, Jul 14, 2017 at 10:02 AM, Nick Allen wrote: > Yes, this is what I did. Pulling this from memory, so I migh

Re: [DISCUSS] Relocate Docker

2017-07-19 Thread Nick Allen
pieces of the code > > base that would fit into this type of area? > > > > -Kyle > > > > On Thu, Jul 13, 2017 at 12:30 PM Laurens Vets wrote: > > > > > On 2017-07-13 09:04, Nick Allen wrote: > > > > Having metron-docker at the top-level of

Re: [DISCUSS] Easing the ramp-up into contributing

2017-07-27 Thread Nick Allen
Good discussion to bring up Justin. All good suggestions on your part. One I'd add is that I'd like to see us improve the "What the heck is Metron?" experience. Right now after looking at our home page or GitHub repo I really don't understand what Metron does and why I should be interested in it

Re: Profiler statistics NaN

2017-08-09 Thread Nick Allen
It seems that you are using the Profiler Client API correctly from the REPL, but you are using it incorrectly in your triage rules. Change your triage rules to match what you ran in the REPL. Correct: PROFILE_GET( "host-talks-to" , "99.191.183.156", PROFILE_FIXED(300, "MINUTES")) Incorrect: PROF

Re: Profiler statistics NaN

2017-08-09 Thread Nick Allen
That is another problem. Isn't the simplest answer, to just change this... "result": "HLLP_CARDINALITY(outcoming)" to this... "result": "outcoming" ? On Wed, Aug 9, 2017 at 3:48 PM Casey Stella wrote: > Ok, so the problem here is that your profile is returning integers > (specifically HLLP

Re: Profiler statistics NaN

2017-08-09 Thread Nick Allen
P object, not a statistics object, so doing a > STATS_MERGE on a bunch of them wouldn't work either. > > On Wed, Aug 9, 2017 at 4:15 PM, Nick Allen wrote: > > > That is another problem. Isn't the simplest answer, to just change > this... > > > >

Re: Profiler statistics NaN

2017-08-09 Thread Nick Allen
lternative to consider. On Wed, Aug 9, 2017 at 4:31 PM Nick Allen wrote: > Oh yeah, duh. Now I'm with you. That would be a good quick hit. > > The current behavior is a little nutty. If there is a list, it only > consumes the first element in the list. I'd expect tha

Re: Profiler statistics NaN

2017-08-09 Thread Nick Allen
ng toward STATS_ADD or STATS_INIT taking a list of > numbers. > > STATS_MERGE seems confusing. > > > > On Wed, Aug 9, 2017 at 4:37 PM, Nick Allen wrote: > > > > > Or even change the behavior of STATS_MERGE, too? If STATS_MERGE gets > raw > > > numbers, it

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Nick Allen
Would it make sense to just 'fix' at a specific version all NPM dependencies, instead of allowing any version after x.y.z ? On Fri, Aug 11, 2017 at 10:34 AM Casey Stella wrote: > I agree; we should have a long-term JIRA for moving to yarn (so confusing > *sigh*) and a tactical JIRA + PR to fix

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Nick Allen
And I suggest that as a short-term (or medium-term) fix, separate from moving wholesale to another dependency mechanism. On Fri, Aug 11, 2017 at 10:36 AM Nick Allen wrote: > Would it make sense to just 'fix' at a specific version all NPM > dependencies, instead of allowing a

Re: Metron Alerts bombing in Travis?

2017-08-11 Thread Nick Allen
ot;how we use it issue", rather than an issue with the tech itself. Do we > have a more compelling reason, or set of reasons, beyond just fixing to > specific version of our dependencies? > > On Fri, Aug 11, 2017 at 10:37 AM, Nick Allen wrote: > > > And I suggest that as a

Re: [DISCUSS] Using Yarn package manager for metron-alerts

2017-08-16 Thread Nick Allen
Thanks for laying this all out for us, Raghu. Based on the built-in support for offline installs and version locking, I think this is a great suggestion. (However unfortunate the namespace collision might be.) On Wed, Aug 16, 2017 at 8:51 AM RaghuMitra Kandikonda < raghumitra@gmail.com

Re: [DISCUSS] Using Yarn package manager for metron-alerts

2017-08-16 Thread Nick Allen
, and I just switched metron-alerts projects to yarn (as a test) and performed an offline install. It was stupid simple.​ On Wed, Aug 16, 2017 at 4:12 PM Nick Allen wrote: > Thanks for laying this all out for us, Raghu. Based on the built-in > support for offline installs and version locking,

Re: [DISCUSS] Metron release 0.4.1

2017-08-22 Thread Nick Allen
Thanks for starting the process, Matt. These are my own open PRs that I would most like to see get in. They all relate to the Profiler. My other outstanding PRs are less important. - https://github.com/apache/metron/pull/705 - https://github.com/apache/metron/pull/707 - https://github.c

Re: [DISCUSS] Synopsis of Community Meeting on 8/22/2017

2017-08-23 Thread Nick Allen
That is actually our plugin. Well, ours is theirs. The Bro Project is getting out of the business of maintaining plugins for the most part. This is one side effect of the new Bro packaging mechanism. The plan is to migrate the code to the new Bro packaging mechanism and for it to live here http

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Nick Allen
+1 I like it all, Otto. You deserve a freakin' medal. On Wed, Aug 23, 2017 at 10:04 AM Otto Fowler wrote: > WRT : regression fixes, I would also like us to consider putting these the > initial 777 to feature branch PR as an option. > > > On August 23, 2017 at 09:56:33, Otto Fowler (ottoback

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Nick Allen
tially be warranted. > > +1 > > Jon > > On Wed, Aug 23, 2017 at 12:38 PM Nick Allen wrote: > > > +1 I like it all, Otto. You deserve a freakin' medal. > > > > > > > > > > > > On Wed, Aug 23, 2017 at 10:04 AM Otto Fowler > >

Re: verify_licenses.sh and commits

2017-08-28 Thread Nick Allen
I see the verify_licenses.sh is set to run "before_install" in Travis. I am wondering why the CI build did not fail if this was a problem? We should open a JIRA so we can track it down, if this wasn't the case. Thanks for correcting. On Sat, Aug 26, 2017 at 11:24 AM Otto Fowler wrote: > Le

Re: Feature Branch: Extension System for Metron and Metron Parsers

2017-08-30 Thread Nick Allen
Yes, I think you still need +1s. The same PR rules apply to the feature branch PRs. The only difference being that as a reviewer/committer I won't expect the same level of quality, documentation, etc to get my +1 for a PR that is destined for a feature branch. And of course, each reviewer/commit

Re: Feature Branch: Extension System for Metron and Metron Parsers

2017-08-30 Thread Nick Allen
ves the other branches in. I don’t > think we expect that pr to get reviewed. > > > > On August 30, 2017 at 10:26:00, Nick Allen (n...@nickallen.org) wrote: > > Yes, I think you still need +1s. The same PR rules apply to the feature > branch PRs. > > The only di

Re: [DISCUSS] Metron release 0.4.1

2017-08-30 Thread Nick Allen
t; > For my PRs, I'd vote for METRON-1122 being in (commit very > imminent). > > I'd very much like METRON-1134 to be in as well. > > > > Beyond that, I'm ok > > > > On Tue, Aug 22, 2017 at 4:37 PM, Nick Alle

Re: [DISCUSS] Feature Branches and updating from Master

2017-08-30 Thread Nick Allen
I think opening PRs is the way to go. We're going to have maybe three kinds of merges when we sync a feature branch with master. - The *good* kind when there are no conflicts. Yay! Easy review, no problems, quick +1. - The *bad* kind when there are conflicts. This will take some level

Maven Exec REPL with all Metron Dependencies

2017-08-31 Thread Nick Allen
Previously, I found that you could run the REPL from the Metron source by running the following command from the ​source root ​ directory. This is useful because it lets you run the REPL with the libraries that will be available ​during Enrichment.​ mvn exec:java \ -Dexec.mainClass="org.apache.m

Re: [DISCUSS] Metron release 0.4.1

2017-08-31 Thread Nick Allen
me work around fixing how we handle config with Ambari that I'd > > >like to see go in. No PR yet, but coming soon. I expect to have this by > > the > > >RC deadline. > > > > > >Mike > > > > > >On Wed, Aug 30, 2017 at 8:57 AM, Nick Allen

Re: Maven Exec REPL with all Metron Dependencies

2017-08-31 Thread Nick Allen
t; doing two -pl statements, but there are these problems. > > While working on this PR, I had a version that *did* package management > and it’s dependencies > and deployment. But I had questions about if it was correct or not given > the class path and hadoop configuration issues etc

Re: [DISCUSS] Metron release 0.4.1

2017-09-05 Thread Nick Allen
gt;>> >>> Thanks, >>> Anand >>> >>> >>> >>> On 9/1/17, 12:41 AM, "Matt Foley" wrote: >>> >>> >Please mark them 0.4.1, as that’s what the community says we want to >>> call the upcoming

<    1   2   3   4   >