Re: Debug enrichment topology

2017-07-31 Thread Simon Elliston Ball
Topology logs appear in the storm worker logs. The easiest way to get to them is through the storm UI, you can also use this when necessary to increase the log level of various packages, which can help to find obscure processing bugs. Simon > On 31 Jul 2017, at 15:29, bharath phatak

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

2017-07-12 Thread Simon Elliston Ball
Looks like you’ve set the profile to purge (expires) every 30 ms, and your period is set to 30 minutes, so the data is being expired long before it has a change to write. Simon > On 12 Jul 2017, at 06:17, Krishna Dhanekula > wrote: > > I have an problem where

Re: Metron Profiler 0.3.0: HbaseBolt not storing data to HBase Instance

2017-07-12 Thread Simon Elliston Ball
kafka.start=WHERE_I_LEFT_OFF > > > > Regards, > > Balakrishna > > > From: Simon Elliston Ball [mailto:si...@simonellistonball.com] > Sent: Wednesday, July 12, 2017 3:28 PM > To: user@metron.apache.org > Subject: Re: Metron Profiler 0.3.0: HbaseBol

Re: How to change Elasticsearch indexing policy

2017-07-14 Thread Simon Elliston Ball
You could change the index data format. One word of caution here though; the last time I saw this done it caused huge problems with locking on ingest against people running queries on the current day’s data and tended to knock recent relevant indexes out of disk cache at the OS level. It might

Re: Threat triage rules using stellar geo enrichment

2017-08-08 Thread Simon Elliston Ball
A much better way of doing this is to run the geo enrichment as part of the regular enrichment process and then just use the output field for the rule. Your config already does this, so your rule is in effect running the same enrichment twice. Just use enrichments.geo.ip_dst_addr.country !=

Re: Question on Windows event log ingest and parse

2017-05-03 Thread Simon Elliston Ball
gt; wrote: > > Correction, deploying the Storm topology is this: > > /usr/metron/$METRON_VERSION/bin/start_parser_topology.sh -z `hostname > -f`:2181 -k `hostname -f`:6667 -s winlogbeat > > > > > > From: Simon Elliston Ball <si...@simonellistonball.com> &

Re: Metron in-memory enrichment

2017-06-19 Thread Simon Elliston Ball
Surely the caching should make this effectively an in memory lookup. Does the stellar enrichment function not use the same clientside caching as the Hbase bolt? Simon > On 19 Jun 2017, at 06:21, Casey Stella wrote: > > In order to do that, the easiest thing to do is to

Re: AWS cloudformation script for Metron deployment

2017-05-23 Thread Simon Elliston Ball
This might be a neater solution than the current ansible build, however, it might also be worth considering something more cloud neutral, since more and more of the project is moving into ambari, blueprints and cloud break might also be an option worth looking at. Simon > On 23 May 2017, at

Re: Build fails - unable to find https://raw.github.com

2017-05-05 Thread Simon Elliston Ball
Is your network or proxy blocking github access for any reason? I’ve seen some corporate environment block github. Simon > On 5 May 2017, at 04:10, Kevin Waterson wrote: > > Yes, seems to fail at the same point each time. > > On Thu, May 4, 2017 at 9:09 AM, Michael

Re: 192.168.138.158 address in yaf index

2017-09-20 Thread Simon Elliston Ball
That sounds like an address from the standard example.pcap used to demo metron capability. In a real deployment you should not run pcap-replay which is what inserts this demo data. Simon > On 21 Sep 2017, at 00:29, Frank Horsfall > wrote: > > Morning all, >

Re: Initial Testing

2017-10-05 Thread Simon Elliston Ball
Try the ambari files view. > On 5 Oct 2017, at 09:24, Syed Hammad Tahir wrote: > > THanks again, also how can I access the snort log via hdfs? Is there any web > based hdfs portal or will I have to sneak into the vagrant VM file system to > access that? > >> On Thu, Oct

Re: Initial Testing

2017-10-05 Thread Simon Elliston Ball
Syed, I would strongly suggest you go through the Squid based tutorial to get an idea of how enrichment and indexing works. See: https://cwiki.apache.org/confluence/display/METRON/Metron+Reference+Application >

[DISCUSS] Dropping support for elastic 2.x

2017-10-04 Thread Simon Elliston Ball
A number of people are currently working on upgrading the ES support in Metron to 5.x (including the clients, and the mpack managed install). Would anyone have any objections to dropping formal support for 2.x as a result of this work? In theory the clients should be backward compatible against

Re: Metron Error in Barematel Installation

2017-10-16 Thread Simon Elliston Ball
This looks like an error in the frontend build. Sometimes this is transient (problems downloading npm packages) so a retry may help. However, we really should be looking at pinning the dependency versions, as this can also be caused by third-party npm packages being updated in the wild and

Re: Metron Error in Barematel Installation

2017-10-16 Thread Simon Elliston Ball
id some dependency got updated... > and its is breaking it. > > On Mon, Oct 16, 2017 at 4:25 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > This looks like an error in the frontend build. Sometimes this is transi

Re: Clearing of data to start over

2017-09-06 Thread Simon Elliston Ball
Multiple Kafka brokers will help a lot. The wizard allows you too add more by using the plus symbol next to Kafka on the master selection screen. After the fact you can add more with the add service button on the hosts screen in ambari. When adding brokers, don't forget to also alter your

Re: Not seeing any Metron alerts.

2017-09-26 Thread Simon Elliston Ball
in Kibana, but I >> don't get a threat.triage.level field which means that either my >> riskLevelRules rules don't trigger or something else goes wrong. >> >> How and where can I look for additional information on why my rules >> might not be working? (Metron UI a

Re: Not seeing any Metron alerts.

2017-09-26 Thread Simon Elliston Ball
yways in the event after I refreshed the fields in > Kibana right? > > On 2017-09-26 09:16, Simon Elliston Ball wrote: > >> There should be, though you may need to update your templates in ES if >> you've got any custom templates there, and make sure you refre

Re: Metron Alerts UI, no alerts

2017-09-28 Thread Simon Elliston Ball
Right now, you can't. I believe we should be taking the lost of index prefixes we use in the ui from the index config via the rest api, we can pull the names from each sensor index config and use that as the prefix in the ui. That way we pickup any new index automatically. Simon > On 28 Sep

Re: Metron Alerts UI, no alerts

2017-09-28 Thread Simon Elliston Ball
nown limitation > to be addressed later. > > >> On September 28, 2017 at 15:29:57, Simon Elliston Ball >> (si...@simonellistonball.com) wrote: >> >> Right now, you can't. I believe we should be taking the lost of index >> prefixes we use in the ui from the index config

Re: Metron Alerts UI, no alerts

2017-09-28 Thread Simon Elliston Ball
It would, yes (that’s what I do at the moment) but you need to rebuild the alerts ui as stands to make that take effect. I expect we’ll get that fixed very shortly. > On 28 Sep 2017, at 22:10, Laurens Vets wrote: > > I didn't know that only the default sensors are shown.

Re: Installation

2017-10-02 Thread Simon Elliston Ball
host actions -> start all components. > On 2 Oct 2017, at 07:22, Syed Hammad Tahir wrote: > > Hi, > > I manually installed every component. Now how do I start them all. I need to > use metron. WHat do I do at this step? > >

Re: Metron Installation error

2017-09-28 Thread Simon Elliston Ball
e python > script it ran before failure. The script which tries to download the packages > >> On Thu, Sep 28, 2017 at 3:23 PM, Simon Elliston Ball >> <si...@simonellistonball.com> wrote: >> It looks like you do not have access to the internet, or at least your >>

Re: Metron Installation error

2017-09-28 Thread Simon Elliston Ball
-yarn > If I do it succesfully then should I do vagrant provision again or anything > else? > > On Thu, Sep 28, 2017 at 3:32 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > Just try a yum install of the package

Re: Metron Installation

2017-10-03 Thread Simon Elliston Ball
All of them. > On 3 Oct 2017, at 11:00, Syed Hammad Tahir wrote: > > WHat services are necessary to run metron? > >

Re: Not seeing any Metron alerts.

2017-09-25 Thread Simon Elliston Ball
the _score field is actually an elastic search matching score field, and is not relevant to metron. You should see the scores in the threat:triage:score field. However, your rules will only be run if the telemetry has is_alert set true, so you should ensure that the enrichment phase sets

Re: Not seeing any Metron alerts.

2017-09-25 Thread Simon Elliston Ball
needs to be set to True. Next to apply the actual scores? > > On 2017-09-25 11:00, Simon Elliston Ball wrote: >> the _score field is actually an elastic search matching score field, >> and is not relevant to metron. You should see the scores in the >> threat:triage:s

Re: Not seeing any Metron alerts.

2017-09-25 Thread Simon Elliston Ball
ot;, > "is_alert := exists(is_alert) && is_alert", > "is_alert := is_alert || (geo_outlier != null && geo_outlier == > true)", > "geo_distance_distr := null" >] > } >}, > > For instance, can the 2nd is_alert lin

Re: Unable to add the hosts

2017-09-25 Thread Simon Elliston Ball
The list says it wants one host per line, you have given it comma separated. > On 25 Sep 2017, at 09:31, kotipalli venkatesh > wrote: > > > Hi All, > > Please help on the below error, Target host, we added nodes and import the > id_rsa file on the main node.

Re: Metron Installation

2017-09-25 Thread Simon Elliston Ball
This looks like it’s probably a timeout. From your other posts it sounds like the machine you’re using is really not up to running the base platform for Metron. I would strongly recommend going for something cloud based. I would also consider using the mpack method on an existing ambari, and

Re: Enable geo enrichment

2017-10-05 Thread Simon Elliston Ball
http://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html Shows you how to configure geo enrichment. Simon > On 5 Oct 2017, at 22:33, Laurens Vets wrote: > >

Re: Enable geo enrichment

2017-10-05 Thread Simon Elliston Ball
es you the relevant info. > On 5 Oct 2017, at 22:36, Simon Elliston Ball <si...@simonellistonball.com> > wrote: > > http://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html > > <http://metron.apache.org/current-book/metron-platform/metron-

Re: event correlation on metron

2017-10-17 Thread Simon Elliston Ball
; that’s what i mean. > what sensor that i need if i want to do this case? > especially when i wanna parse some host logs into metron enrichment and > indexing > >> On Wed, 18 Oct 2017 at 01.03 Simon Elliston Ball >> <si...@simonellistonball.com> wrote: >> W

Re: event correlation on metron

2017-10-17 Thread Simon Elliston Ball
What you want to do in this setting is just TailFile, the just push to Kafka. The grok piece is more efficiently handled in the Metron grok parser. Push to a kafka topic named for your sensor, then setup a sensor (a parser topology to do the grok parsing and any transformation you need). Each

Re: Snort

2017-10-19 Thread Simon Elliston Ball
I would recommend just using a text editor if you’re not familiar with sed. To solve your sed problem… sed -i.bak "s/ansibleSkipTags=.*/ansibleSkipTags=\'quick_dev\'/" metron-deployment/vagrant/full-dev-platform/Vagrantfile sed -i means run the sed command (in this case a find replace)

Re: Sizing of components proportional to EPS

2017-10-17 Thread Simon Elliston Ball
To an extent it very much depends on the use case. I have seen over a million EPS on a six node cluster for pcap and basic net flow. If you add a lot of complex enrichment and profiling that will obviously increase the load. Tuning the components for the workload can also make a significant

Re: multiple pattern grok parser in 1 file

2017-10-23 Thread Simon Elliston Ball
the issue might be that your patterns flat out don’t match the logs. Simon > On 23 Oct 2017, at 10:36, tkg_cangkul <yuza.ras...@gmail.com> wrote: > > Hi Simon, > > I've tried your suggestion but i have an error msg like below : > > > > On 23/10/17

Re: multiple pattern grok parser in 1 file

2017-10-23 Thread Simon Elliston Ball
That is not valid grok. Pattern names should be unique in the grok. What you probably mean is something like: AUTHLOG1 %{NUMBER:timestamp} %{SYSLOGHOST:syslog_host} %{DATA:syslog_program}(?:\[%{POSINT}\])?: %{WORD:login} password for %{USERNAME:username} from %{IP:ip} %{GREEDYDATA} AUTHLOG2

Re: Not able to run metron.

2017-11-22 Thread Simon Elliston Ball
Sorry, you’re right, you do need ansible. Make sure the version is EXACTLY the version in the docs. Simon > On 22 Nov 2017, at 13:03, Otto Fowler wrote: > > You DO need ansible for full_dev deployment. > You do need Docker installed and running > > > > On November

Re: Not able to run metron.

2017-11-22 Thread Simon Elliston Ball
Otto Fowler (ottobackwa...@gmail.com > <mailto:ottobackwa...@gmail.com>) wrote: > >> I build on mac, and have : >> >> -- >> node >> v6.10.2 >> -- >> npm >> 3.10.10 >> >> for my node versions. >> >> >&

Re: Not able to run metron.

2017-11-22 Thread Simon Elliston Ball
You shouldn’t need ansible for the full-dev build, but you will need maven, docker and an up-to-date nodejs and npm package to do the actual build. I would recommend against using the OS provided nodejs and go with the packages from nodesource instead. The full-dev build is also the best

Re: Snort enrichment issue

2017-11-17 Thread Simon Elliston Ball
Did you setup and load the geo enrichment database? https://metron.apache.org/current-book/metron-platform/metron-data-management/index.html#GeoLite2_Loader Also, we can’t really see the

Re: ML in Metron

2017-11-21 Thread Simon Elliston Ball
Use MaaS: http://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html > On 21 Nov 2017, at 11:43, Syed Hammad Tahir wrote: > > HI all, > > I have

Re: analytics exchange platform

2017-11-15 Thread Simon Elliston Ball
The analytics exchange concept is not really part of Apache Metron, but some commercial offerings include it. In terms of Metron itself, are you maybe thinking about Model as a Service: http://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html

Re: machine learning libraries supported

2017-12-07 Thread Simon Elliston Ball
I would recommend starting out with something like Spark, but the short answer is that anything that will run inside a yarn container, so the answer is most ML libraries. Using Spark to train models on the historical store is a good bet, and then using the trained models with model as a

Re: machine learning libraries supported

2017-12-07 Thread Simon Elliston Ball
gt; Is there a performance problem or how would you justify that phrase? > > thanks > > Le 07/12/2017 à 13:55, Simon Elliston Ball a écrit : >> I would recommend starting out with something like Spark, but the short >> answer is that anything that will run inside a yarn cont

Re: Basic analysis

2017-12-06 Thread Simon Elliston Ball
Yes. Consider a zeppelin notebook, or kibana dashboard for this. If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics. Simon > On 6 Dec 2017, at 07:42, Syed Hammad Tahir

Re: Basic analysis

2017-12-06 Thread Simon Elliston Ball
iate > as well? > > > On December 6, 2017 at 07:06:30, Simon Elliston Ball > (si...@simonellistonball.com <mailto:si...@simonellistonball.com>) wrote: > >> Yes. Consider a zeppelin notebook, or kibana dashboard for this. >> >> If you want to us

Re: Basic analysis

2017-12-06 Thread Simon Elliston Ball
providers I believe to add more samples of both dashboards and use cases. Simon > On 6 Dec 2017, at 14:12, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Thanks Simon > > > On December 6, 2017 at 09:11:50, Simon Elliston Ball > (si...@simonellistonball.com <mail

Re: Basic analysis

2017-12-06 Thread Simon Elliston Ball
<https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard> Should at least get us started. Simon > On 6 Dec 2017, at 14:00, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Links? > > > On December 6, 2017 at 08:18:23, Simon Elliston Ball >

Re: Basic analysis

2017-12-06 Thread Simon Elliston Ball
> The issue is the requirement for people on the user list to go to the source. > > > On December 6, 2017 at 09:16:39, Simon Elliston Ball > (si...@simonellistonball.com <mailto:si...@simonellistonball.com>) wrote: > >> No problem, I’ll grant you it’s not in the most i

Re: Basic analysis

2017-12-06 Thread Simon Elliston Ball
> On Wed, Dec 6, 2017 at 7:35 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > Agreed… for the users list I would just say use the Install Notebooks action, > and look at the squid example on the wiki, but since it was y

Re: Kibana Error

2017-10-25 Thread Simon Elliston Ball
ambari > after changing heapsize. Now doing it via console > > > > On Wed, Oct 25, 2017 at 5:13 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > That just shows running, not health. The problem is that it is not

Re: Kibana Error

2017-10-25 Thread Simon Elliston Ball
t; > > > On Wed, Oct 25, 2017 at 3:45 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > I strongly suggest you spend some time learning about elastic search and some > of the basic components. This is not a bug, it’

Re: Kibana Error

2017-10-25 Thread Simon Elliston Ball
s this > > > > On Wed, Oct 25, 2017 at 5:07 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simonellistonball.com>> wrote: > Did you check the elastic service was running and healthy with the health > checks. Try a few of the quick links

Re: SysLog Parser in Metron

2017-10-25 Thread Simon Elliston Ball
Short answer: grok parsers. Longer answer: syslog is more a transport, not just a log format, so it encapsulates a wide variety of data sources. Your best bet is probably to use NiFi to listen for syslog from a remote host (ListenSyslog) and then route each application in the syslog to a

Re: Kibana Error

2017-10-25 Thread Simon Elliston Ball
Your elastic search instance has died. Try given it more heap size in the elastic section on ambari. > On 25 Oct 2017, at 09:16, Syed Hammad Tahir wrote: > > When I try to open node1:5000 I see this. > > > > What could be the problem and its solution?

Re: Kibana Error

2017-10-25 Thread Simon Elliston Ball
cs16...@itu.edu.pk> wrote: >>> SHould I do it from here? If yes then please guide me how to >>> >>> >>> >>>> On Wed, Oct 25, 2017 at 1:17 PM, Simon Elliston Ball >>>> <si...@simonellistonball.com> wrote: >>>> Your elastic search instance

Re: metron vs ossec

2017-12-21 Thread Simon Elliston Ball
In many ways it’s a matter of scale. OSSIM is a kind of lite version of AlienVault, and used by them. I’ve seen people move from an OSSIM architecture to Metron specifically to get better scaling, things like PCAP capabilities etc. but also retain the OSSEC agents to handle endpoint and

Re: Metron Version

2018-01-04 Thread Simon Elliston Ball
Are the logs you’re sending with syslog in CEF format? You will note that the CEF sensor uses the CEF parser, which means unless your logs are in CEF format, they will fail to parse and be dropped into the error index (worth checking the error index in kibana via the Metron Error Dashboard.

Re: Metron Rest Kerberos -- Kafka topic ACL

2018-01-10 Thread Simon Elliston Ball
The ansible roles and playbooks included with Metron install Ambari to handle the setup of the Metron and the Hadoop, Kafka etc. components, so yes. > On 10 Jan 2018, at 03:18, varsha mordi wrote: > > Can Ambari UI work with Ansible? > > On Wed, Jan 10, 2018 at

Re: Metron Reference Application (Profiling Your Streams Fails)

2018-01-15 Thread Simon Elliston Ball
Looks like a docs typo on the wiki: What you need is CONFIG_PUT(“PROFILER”, profilerConfig) Simon > On 15 Jan 2018, at 10:45, Farrukh Naveed Anjum > wrote: > > Can you help on this ? > > On Mon, Jan 15, 2018 at 3:42 PM, Farrukh Naveed Anjum >

Re: SysLog using CEF Parser (RSysLogs)

2018-01-22 Thread Simon Elliston Ball
Are there any errors in the logs for the indexing bolt? I would expect the errors are probably at the elastic ingest point, and probably caused by an incorrect elastic template for the CEF data. Simon > On 22 Jan 2018, at 08:24, Farrukh Naveed Anjum > wrote: > >

Re: Define a function that can be used in Stellar

2018-01-17 Thread Simon Elliston Ball
there any example regarding > adding a Stellar function in Java? Hopefully, we don't need to rebuild the > corresponding modules for this? > > Cheers, > Ali > > On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball > <si...@simonellistonball.com <mailto:si...@simo

Re: Define a function that can be used in Stellar

2018-01-17 Thread Simon Elliston Ball
At present you can certainly create custom stellar functions in Java. I’m guessing however that what you’re looking to do is create a kind of function that combines a number of stellar functions to avoid repetition, or to ensure consistency of certain parameters for example. Is that what you’re

Re: Metron User Community Meeting Call

2018-01-26 Thread Simon Elliston Ball
This is going to be a really exciting call. Looking forward to seeing how the GCR Canary sings :) I’m going to volunteer https://hortonworks.zoom.us/my/simonellistonball as a location for the meeting. I would also support the idea of a quick poll on what people are doing with Metron, and

Re: Apache Metron functions implementation

2018-02-02 Thread Simon Elliston Ball
Hi Helder, It is very much possible, and very easy to create your own functions and models on top of Metron. There are two main ways in which you would do this, depending on the type of use case you’re looking at. Metron uses a language called Stellar as part of the enrichment stage (and

Re: Define a function that can be used in Stellar

2018-02-02 Thread Simon Elliston Ball
dy been implemented in Metron that has a > config file associated with it? I am trying to get an idea of how it works. > > On 3 Feb. 2018 00:44, "Simon Elliston Ball" <si...@simonellistonball.com > <mailto:si...@simonellistonball.com>> wrote: > Depends how you write t

Re: Define a function that can be used in Stellar

2018-02-02 Thread Simon Elliston Ball
n.apache.org>> > Subject: Re: Define a function that can be used in Stellar > > > > > > > > If something we have already does not fit the bill, I would recommend > creating that function in Java. Since you described it as "a bit complex"

Re: Define a function that can be used in Stellar

2018-02-02 Thread Simon Elliston Ball
l one it will only read it one time and it will be > available in memory? > > On 2 Feb. 2018 21:53, "Simon Elliston Ball" <si...@simonellistonball.com > <mailto:si...@simonellistonball.com>> wrote: > Shouldn’t be. The one this I would point out though is

Re: HBase enrichment vs Stellar enrichment for HBase look up

2018-02-02 Thread Simon Elliston Ball
There shouldn’t be. Both run through the same kind of bolt-side caching, so you should be able to use the Stellar version, and in fact that’s the general direction the project is heading. We haven’t quite deprecated the plain HBase Bolt… but Stellar is definitely the preferred option. Simon

Re: Stellar post-parsing transformation conditional statement

2018-02-08 Thread Simon Elliston Ball
You either want a MAP_GET in your IF or a match statement in there I expect. See the match statement at https://github.com/apache/metron/blob/master/metron-stellar/stellar-common/README.md under core functions (it’s relatively new) Simon Sent from my iPhone > On 9 Feb 2018, at 03:55, Ali

Re: Error when trying to install Apache Metron CentOS7

2018-02-14 Thread Simon Elliston Ball
To be honest, rather than messing about with grub for this, I would follow the alternative route outlines in the wiki page. To be even more honest, I wouldn’t use that method from the wiki and would probably go with something like the full dev VM platform if you’re looking to do development

Re: elasticsearch template question.

2018-02-07 Thread Simon Elliston Ball
Hi Laurens, In Metron all fields tend to get flattened into an un-nested structure of keys and values. Some of the keys do represent a flattened tree structure (for example our standard enrichment fields). The reason for this is essentially ingest speed for nested documents in lucene based

Re: CentOS and Ubuntu

2018-02-07 Thread Simon Elliston Ball
Not particularly. The centos builds seem to be used by more people on dev, probably because they’ve been around for longer, and so are arguably more tested. The area where it’s most likely to be relevant is in the install of repos for ES and potentially the fastcapa pcap probe (don’t quote me

Re: Best Metron version for development

2018-02-15 Thread Simon Elliston Ball
The full dev platform may be the easiest to test things like that on. It can be a little brittle if you’re running it in limited RAM, but it also has things like the sensor-stubs, which provides an easy means to fake up some input traffic. That may be useful for your development and testing.

Re: Define a function that can be used in Stellar

2018-01-17 Thread Simon Elliston Ball
; >>> Yeah, I agree. It will be much easier to define functions on the fly and >>> use them afterwards. It could be defined as Lambda or custom function. >>> >>> Regards, >>> Ali >>> >>> >>> >>>> On Wed

Re: Some Metron Alerts UI questions

2018-01-22 Thread Simon Elliston Ball
Hi Laurens, A few quick answers inline… Simon > On 20 Jan 2018, at 00:37, Laurens Vets wrote: > > Hi list, > > I have some general Alerts UI questions/comments/remarks, I hope you don't > mind :) I'm using the UI that's part of Metron 0.4.2. These apply to my > specific

Re: Indexing Bolt Error

2018-01-24 Thread Simon Elliston Ball
Yes, configure your indexing. https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html Note it’s a warning, not an error, that default values are being used because you do not have a

Re: Metron Indexing Service Failing Shortly After Start

2018-02-27 Thread Simon Elliston Ball
Anything appearing on the indexing topic in kafka? Anything in the logs for the indexing topology in Storm UI? Master isn’t always the best place to start, might be worth sticking to a released build to kick the tyres. Simon > On 27 Feb 2018, at 17:38, David McGinnis

Re: Alerts Not Being Generated?

2018-03-01 Thread Simon Elliston Ball
Hi David, One quick thing just in case, is_alert, not is_alarm. That said that should not affect what’s in the alerts ui. You should see data from your geo source as well (whatever you called it). It is possible there may be a problem with your elastic template. You might be interested in