In the Vagrantfile for full-dev, edit the line that starts with ansibleSkipTags (this line <https://github.com/apache/metron/blob/master/metron-deployment/vagrant/full-dev-platform/Vagrantfile#L20>) to be exactly the following:
ansibleSkipTags='quick_dev' Jon On Thu, Oct 19, 2017 at 7:59 AM Syed Hammad Tahir <[email protected]> wrote: > Should I edit the vagrant file using text editor and what exactly should I > edit there? > > On Thu, Oct 19, 2017 at 3:54 PM, Simon Elliston Ball < > [email protected]> wrote: > >> I would recommend just using a text editor if you’re not familiar with >> sed. To solve your sed problem… >> >> sed -i.bak "s/ansibleSkipTags=.*/ansibleSkipTags=\'quick_dev\'/" >> metron-deployment/vagrant/full-dev-platform/Vagrantfile >> >> sed -i means run the sed command (in this case a find replace) inplace >> on the file, the text following the -i is the name to append to a backup >> version (ie the original file unchanged). >> >> Metron does tend to assume a good knowledge of linux admin, you’ll find >> we have a lot of shell gurus in the community, but if you’re struggling >> with this, maybe a simple text editor would be easier. All you’re trying to >> do here is change a config value. >> >> Simon >> >> On 19 Oct 2017, at 11:46, Syed Hammad Tahir <[email protected]> wrote: >> >> Ran it without -i swtich, gives this: >> >> <image.png> >> >> On Thu, Oct 19, 2017 at 2:56 PM, [email protected] <[email protected]> >> wrote: >> >>> The sed command is falling. It's written for a Mac so it will need an >>> alteration to be portable. Run it without the '' after -i, from >>> ~/metron-master >>> >>> Jon >>> >>> On Thu, Oct 19, 2017, 04:07 Syed Hammad Tahir <[email protected]> >>> wrote: >>> >>>> I did what this guide said to install the original sensor: >>>> >>>> https://github.com/apache/metron/tree/master/metron-deployment/roles/sensor-stubs >>>> >>>> Still didnt work. How do I install snort into this? >>>> >>>> >>>> On Thu, Oct 19, 2017 at 10:26 AM, Syed Hammad Tahir < >>>> [email protected]> wrote: >>>> >>>>> Maybe I did something wrong >>>>> >>>>> <image.png> >>>>> >>>>> On Thu, Oct 19, 2017 at 6:03 AM, Syed Hammad Tahir < >>>>> [email protected]> wrote: >>>>> >>>>>> Ok, thankyou. It will install all the sensors (bro, snort etc) ? >>>>>> >>>>>> On Thu, Oct 19, 2017 at 12:30 AM, [email protected] <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> When you set up full dev if you remove the sensors skip tag it will >>>>>>> set up snort for you. I have a sed one liner in my bro security patch >>>>>>> pr >>>>>>> to do this, just need to do it before vagrant up. >>>>>>> >>>>>>> sed -i '' "s/ansibleSkipTags=.*/ansibleSkipTags=\'quick_dev\'/" >>>>>>> metron-deployment/vagrant/full-dev-platform/Vagrantfile >>>>>>> cd metron-deployment/vagrant/full-dev-platform/ >>>>>>> vagrant up >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On Wed, Oct 18, 2017, 14:51 Syed Hammad Tahir <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I followed this guide exactly: >>>>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68718548 >>>>>>>> >>>>>>>> And the did vagrant up in full-development-platform folder. And >>>>>>>> Snort is not installed because when I type snort -v in vagrant ssh, it >>>>>>>> returns error of not able to find the snort command. >>>>>>>> >>>>>>>> On Wed, Oct 18, 2017 at 10:12 PM, Laurens Vets <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Syed, >>>>>>>>> >>>>>>>>> I was under the impression that you installed the full-dev >>>>>>>>> environment? If so, snort should already be installed... >>>>>>>>> >>>>>>>>> On 2017-10-18 09:45, Syed Hammad Tahir wrote: >>>>>>>>> >>>>>>>>> It has become a mess. Apparently snort is released for centos 7 >>>>>>>>> whereas metron one is centos 6.8. Whenever I try to install snort it >>>>>>>>> gives >>>>>>>>> me this: >>>>>>>>> >>>>>>>>> <image.png> >>>>>>>>> >>>>>>>>> On Wed, Oct 18, 2017 at 5:58 PM, Nick Allen <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Just use those as a guide to run the commands yourself. >>>>>>>>>> >>>>>>>>>> On Wed, Oct 18, 2017 at 7:23 AM Syed Hammad Tahir < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> please help me install the snort in metron. I tried doing it the >>>>>>>>>>> normal way but i cant install the libraries >>>>>>>>>>> >>>>>>>>>>> On Wed, Oct 18, 2017 at 10:35 AM, Syed Hammad Tahir < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> ok, This is the snort.yml file >>>>>>>>>>>> >>>>>>>>>>>> <image.png> >>>>>>>>>>>> >>>>>>>>>>>> Do I need to run these commands myself or how do I put these >>>>>>>>>>>> yml files into play? >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 17, 2017 at 9:44 PM, Syed Hammad Tahir < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I am so noob in all of this. I am using full-dev vm metron >>>>>>>>>>>>> install to do my research. So I have 2 options to install snort: >>>>>>>>>>>>> as per my >>>>>>>>>>>>> understanding >>>>>>>>>>>>> >>>>>>>>>>>>> 1- Install it in a usual way (like that on a regular linux >>>>>>>>>>>>> machine) and then make its kafka topic >>>>>>>>>>>>> >>>>>>>>>>>>> 2- Use ansible role to do all of that. Read the content of >>>>>>>>>>>>> those yml files given in main.yml to understand the procedure? >>>>>>>>>>>>> >>>>>>>>>>>>> Which one do you suggest? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Oct 17, 2017 at 9:22 PM, Nick Allen < >>>>>>>>>>>>> [email protected]>wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> No special commands. Install and configure Snort however you >>>>>>>>>>>>>> like and get those logs into a Kafka topic. Metron is >>>>>>>>>>>>>> completely agnostic >>>>>>>>>>>>>> to how sensor telemetry lands in Kafka. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We also have an Ansible role that will install Snort along >>>>>>>>>>>>>> with a simple mechanism to transport its logs to Kafka. This is >>>>>>>>>>>>>> only >>>>>>>>>>>>>> useful for development environments; not a production install. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Using the Ansible role directly may be beyond the knowledge >>>>>>>>>>>>>> level of some. I only offer this as a guide that you can use to >>>>>>>>>>>>>> follow >>>>>>>>>>>>>> along and manually install it yourself. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://github.com/apache/metron/blob/master/metron-deployment/roles/snort/tasks/main.yml >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> If you are not familiar with how Ansible roles are defined, >>>>>>>>>>>>>> just start at the main.yml, then follow through each of the >>>>>>>>>>>>>> other files as >>>>>>>>>>>>>> they are included. It is pretty readable once you get use to >>>>>>>>>>>>>> the layout. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 12:05 PM, Syed Hammad Tahir < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ok, Now I get it. Now should I install snort in vagrant ssh >>>>>>>>>>>>>>> in the normal way snort is usually install on a linux distro or >>>>>>>>>>>>>>> do I need >>>>>>>>>>>>>>> to run some special commands again? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 7:45 PM, Nick Allen < >>>>>>>>>>>>>>> [email protected]>wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In the Full Dev environment, Snort is not installed. We >>>>>>>>>>>>>>>> install "Sensor Stubs" which is just a mechanism that >>>>>>>>>>>>>>>> continually replays >>>>>>>>>>>>>>>> canned telemetry logs repetitively to mimic real sensors. We >>>>>>>>>>>>>>>> have to do >>>>>>>>>>>>>>>> this because of resource constraints when running all of >>>>>>>>>>>>>>>> Metron on a single >>>>>>>>>>>>>>>> VM. See the following for more information. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://github.com/apache/metron/tree/master/metron-deployment/roles/sensor-stubs >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 10:16 AM, Syed Hammad Tahir < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> yes,, but when i do snort -v in vagrant ssh console it >>>>>>>>>>>>>>>>> says snort isnt installed where as it can be seen working in >>>>>>>>>>>>>>>>> metron. Due to >>>>>>>>>>>>>>>>> that reason I am confused because James Sirota said to >>>>>>>>>>>>>>>>> install snort. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 7:05 PM, Nick Allen < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> From Metron's perspective, Snort is just another sensor. >>>>>>>>>>>>>>>>>> Snort is installed, managed and executed completely >>>>>>>>>>>>>>>>>> independent of Metron >>>>>>>>>>>>>>>>>> itself. As with any sensor, you are responsible for getting >>>>>>>>>>>>>>>>>> the telemetry >>>>>>>>>>>>>>>>>> produced by Snort into Kafka. Metron can then consume that >>>>>>>>>>>>>>>>>> telemetry from >>>>>>>>>>>>>>>>>> Kafka and do wonderful things with it. :) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 4:00 AM, Syed Hammad Tahir < >>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> And I am sorry about one confusion but isnt snort >>>>>>>>>>>>>>>>>>> builtin into the metron framework? If so then cant we >>>>>>>>>>>>>>>>>>> access that snort and >>>>>>>>>>>>>>>>>>> do the tasks you mentioned earlier? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 11:39 AM, Syed Hammad Tahir < >>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks for the support. Can it be performed both on >>>>>>>>>>>>>>>>>>>> dumped log and real time data? >>>>>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Tue, Oct 17, 2017 at 1:02 AM, James Sirota < >>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> What I mean is that you should install snort, load the >>>>>>>>>>>>>>>>>>>>> appropriate Snort rules for your use case, set Snort to >>>>>>>>>>>>>>>>>>>>> log to a directory, >>>>>>>>>>>>>>>>>>>>> and send traffic to the network interface where Snort is >>>>>>>>>>>>>>>>>>>>> listening. That >>>>>>>>>>>>>>>>>>>>> will produce Snort log files. Then you can push the >>>>>>>>>>>>>>>>>>>>> contents of Snort logs >>>>>>>>>>>>>>>>>>>>> either to Kafka using NiFi (preferred) or using Kafka >>>>>>>>>>>>>>>>>>>>> utilities such as >>>>>>>>>>>>>>>>>>>>> command line producer. This should be pushed to a Kafka >>>>>>>>>>>>>>>>>>>>> topic called Snort >>>>>>>>>>>>>>>>>>>>> where each message is a log line of the Snort file. Does >>>>>>>>>>>>>>>>>>>>> that make sense? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> James >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 11.10.2017, 23:08, "Syed Hammad Tahir" < >>>>>>>>>>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> You mean that I must start snort from terminal by >>>>>>>>>>>>>>>>>>>>> doing snort -v and then push it to kafka topic? I need to >>>>>>>>>>>>>>>>>>>>> start snort in >>>>>>>>>>>>>>>>>>>>> packet capture mode. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Tue, Oct 10, 2017 at 9:52 PM, James Sirota < >>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yes, you can use Snort. Metron can consume Snort >>>>>>>>>>>>>>>>>>>>> telemetries out of the box. You have to setup Snort on >>>>>>>>>>>>>>>>>>>>> your own and push >>>>>>>>>>>>>>>>>>>>> the output into a kafka topic (most likely using NiFi). >>>>>>>>>>>>>>>>>>>>> From there on you >>>>>>>>>>>>>>>>>>>>> can use the output of Snort in Metron. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 10.10.2017, 00:48, "Syed Hammad Tahir" < >>>>>>>>>>>>>>>>>>>>> [email protected]>: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Can I use snort in packet capture mode with metron? By >>>>>>>>>>>>>>>>>>>>> default it works in IDS mode only. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ------------------- >>>>>>>>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> James Sirota >>>>>>>>>>>>>>>>>>>>> PMC- Apache Metron >>>>>>>>>>>>>>>>>>>>> jsirota AT apache DOT org >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ------------------- >>>>>>>>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> James Sirota >>>>>>>>>>>>>>>>>>>>> PMC- Apache Metron >>>>>>>>>>>>>>>>>>>>> jsirota AT apache DOT org >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>> >>>>>>> Jon >>>>>>> >>>>>> >>>>>> >>>>> >>>> -- >>> >>> Jon >>> >> >> > -- Jon
