Re: Issues with indexing topology

ut. Ryan On Mon, Jul 31, 2017 at 12:02 PM, Guillem Mateos <bbguil...@gmail.com> wrote: > Hi, > > I'm facing an issue like the one Christian Tramnitz and Ryan Merriman > discussed in May. > > I have a Metron deployment using 0.4.0 on 10 nodes. The indexing topology >

Re: Storm indexing topology possible memory issues

Laurens, 2 nodes with 32G of RAM is really small considering all the different components included with Metron. Assuming this is a demo or POC cluster, otherwise it's WAY under-sized. This isn't specific to Metron by the way, it applies to any HDP (or similar distribution) cluster. You will

Re: Metron REST fails to start with Ambari, works via CLI

Does this error happen on start or restart? The bug I fixed only happened on restart. On Tue, Jul 11, 2017 at 5:13 PM, Nick Allen wrote: > Yes, I have seen this. It is a bug. I believe Ryan submitted a fix in one > of his open PRs. He can chime in with the exact one. > >

Re: Issues with indexing topology

something > slightly different than committing. So you do ack a message and you commit > it also, but it's not exactly the same. Am I right? > > Thanks > > 2017-07-31 19:40 GMT+02:00 Ryan Merriman <merrim...@gmail.com>: > >> Guillem, >> >> I thi

Re: ES silently stops indexing after 10k messages

Christian, We happened to run into this exact situation a couple days ago while tuning the indexing topology. In our case we were testing HDFS write performance and ES wasn't involved at all. We eventually tracked it down to a bad topic and we suspect the offsets were messed up somehow.

Re: Metron in-memory enrichment

already increased the cache value. However, clearly, the HBase > enrichment is a bottleneck for the entire enrichment topology. The average > latency for that part is about 30-40 ms, but the latency for other parts in > total is less than 0.5 ms. > > On Mon, Jun 19, 2017 at 11:45 PM, Ry

Re: Grok Parser issues

Yes On Wed, Sep 13, 2017 at 10:02 AM, Frank Horsfall < frankhorsf...@cunet.carleton.ca> wrote: > Do you mean this one on port 4200? > > > > > > *From:* Ryan Merriman [mailto:merrim...@gmail.com] > *Sent:* Wednesday, September 13, 2017 10:53 AM > *To:* user@metron

Re: Metron Installation

equisites and not a guide, can I > find a full guide somewhere to install the full dev environment? > > On Fri, Sep 15, 2017 at 7:10 PM, Ryan Merriman <merrim...@gmail.com> > wrote: > >> If you are just looking to demo it (which I assume is true because you're >> installi

Re: Metron Installation

48 > > On Fri, Sep 15, 2017 at 8:07 PM, Ryan Merriman <merrim...@gmail.com> > wrote: > >> I'm not sure, have only built it with Mac OS. Others may have guidance >> for other OSs. >> >> On Fri, Sep 15, 2017 at 10:01 AM, Syed Hammad Tahir <mscs16...@itu.e

Re: Metron Alerts UI not working (anymore)

overview > of API actions. But how can actually log on? > > On 2017-09-27 14:55, Laurens Vets wrote: > > How can I log into the Swagger UI? > > On 2017-09-27 14:38, Ryan Merriman wrote: > > Nevermind it is proxying to metron1, I see it now. Please disregard. Are > you able

Re: Hello

You need to update node and npm. You are on really old versions. Ryan > On Nov 20, 2017, at 9:45 AM, Nick Allen wrote: > > ​Ok. Just to clarify, you reported two bugs. Both of which are related to > building Metron. I think in METRON-1318 [1] you did not have JAVA_HOME

Re: [DISCUSS] Deprecating metron-api

. You might want > to cc user@ as well, to get a broader set of input for the "are people > using it?" question. > > On Fri, Jun 29, 2018 at 5:21 PM Ryan Merriman wrote: > > > We are currently working on adding pcap query capabilities to the Alerts > UI > > as

Re: Installing Metron 0.4.1

Thanks Otto you beat me to it. Was this not added to our documentation? On Tue, Jan 9, 2018 at 10:50 AM, Otto Fowler wrote: > As answered on irc, updating gcc got Tarik by this. Laurens FTW! > > > On January 9, 2018 at 11:24:47, Tarik Courdy (tarik.cou...@gmail.com) >

Re: Location of Quickstart "full dev platform"

https://github.com/apache/metron/tree/master/metron-deployment/development/centos6 That was changed recently and the wiki hasn't been updated yet. Your best bet is to rely on the READMEs which are always up to date. On Tue, Feb 6, 2018 at 3:53 PM, Jack Hamm wrote: > Hi

Re: No data in HDFS at /apps/metron/indexing/indexed after complete deployment of Full Development VM

I think you need to go one level deeper, those are directories. Here is what I see in my dev environment: [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed Found 2 items drwxrwxr-x - storm hadoop 0 2018-02-07 01:20 /apps/metron/indexing/indexed/bro drwxrwxr-x - storm hadoop

Re: Logging Into Metron UIs On Kerberized Cluster?

d the two UIs as well as the REST. I > am still not able to log in using these credentials with a "login failed" > error. > >> On Wed, Feb 21, 2018 at 5:24 PM, Ryan Merriman <merrim...@gmail.com> wrote: >> Ah I see. Yes the tables only get created automatic

Re: Logging Into Metron UIs On Kerberized Cluster?

> > Note I'm using postgres and not mysql (which seem to be the default for > Metron). Is there a chance this could be causing an issue? > > On Wed, Feb 21, 2018 at 1:49 PM, Ryan Merriman <merrim...@gmail.com> > wrote: > >> Did you create the metronrest database and permiss

Re: Logging Into Metron UIs On Kerberized Cluster?

ldn't have any effect. >> >> This is not on a production system, but it is on an Azure cluster, as >> preparation for installation onto a production system, so any fix we do >> here should be able to be done on a production system, in case the same >> issue comes up. >>

Re: Logging Into Metron UIs On Kerberized Cluster?

create those > tables and just currently isn't? > > On Wed, Feb 21, 2018 at 1:37 PM, Ryan Merriman <merrim...@gmail.com> > wrote: > >> David, >> >> Authentication in our UIs is not as seamless as it should be yet. >> Kerberos authentication is on the road map.

Re: Logging Into Metron UIs On Kerberized Cluster?

David, Authentication in our UIs is not as seamless as it should be yet. Kerberos authentication is on the road map. For now you need to load your credentials into the appropriate security RDMBS tables. Instructions are in the REST README:

Re: Unable to see alerts in metron alert UI with solr

The first thing I would do is figure out which indices are included in the Alerts UI. If you open the network tab in your browser there should be a "search" request sent to REST. There is an "indices" property that will have a list of indices include in the search. If it's empty ("[]") then

Re: Metron Indexing Service Failing Shortly After Start

nt import ( > File "/usr/lib/python2.7/site-packages/urllib3/packages/six.py", line > 198, in load_module > return sys.modules[fullname] > AttributeError: 'NoneType' object has no attribute 'modules' > > On Tue, Feb 27, 2018 at 1:12 PM, Ryan Merriman <merrim...@gmail.com&

Re: Metron Indexing Service Failing Shortly After Start

; File "/usr/lib/python2.7/site-packages/requests/packages/__init__.py", >> line 61, in load_module >> if name in sys.modules: >> AttributeError: 'NoneType' object has no attribute 'modules' >> >> Perhaps there is a missing component that needs to be ins

Re: Metron rest db issues

If I had to guess the dev profile is causing your problems. The dev profile automatically loads a couple test users for development so if you are adding the same user(s) separately then I would expect the error you are seeing. A couple of ideas: - skip the step in the instructions to add users

Re: batch indexing in JSON format

The short answer is no. Offline conversion to other formats (as you describe) is a better approach anyways. Writing to a Parquet/ORC file is more compute intensive than just writing JSON data directly to HDFS and not something you need to do in real-time since you have the same data available