Thanks a lot Michael for your help. Will explore further. On Wed, Apr 10, 2019, 3:37 AM Michael Miklavcic <michael.miklav...@gmail.com> wrote:
> That table should have been created by default as part of the Ambari > installation of Metron via our MPack - > https://github.com/apache/metron/tree/master/metron-deployment#how-do-i-deploy-metron-with-ambari. > You shouldn't have to worry about this at all as an end user, but here is > where it happens if you're curious- > https://github.com/apache/metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py#L230 > > I believe that table will be empty by default. @Ryan Merriman, do you > happen to know if we currently save user UI data there? > > On Tue, Apr 9, 2019 at 2:14 AM Hema malini <nhemamalin...@gmail.com> > wrote: > >> Hi Michael, >> >> Issue got resolved after I manually created user settings table in hbase. >> There are no contents in that table though it is working. Where are the >> records getting stored then for alerts UI. Where can I find the records in >> HDFS. >> >> Thanks and regards, >> Hema >> >> On Tue, Apr 9, 2019, 1:12 PM Hema malini <nhemamalin...@gmail.com> wrote: >> >>> Hi Michael, >>> >>> Sorry just noticed the error in metron rest logs - Table 'user settings' >>> was not found. Do we have to create that hbase table . Where to find the >>> hbase tables created. I could see only two namespace in hbase - default and >>> hbase. No tables created in that. Do I have to run metron rest in dev >>> profile. >>> >>> Thanks & Regards >>> Hema >>> >>> On Tue, Apr 9, 2019, 12:44 PM Hema malini <nhemamalin...@gmail.com> >>> wrote: >>> >>>> Hi Michael, >>>> >>>> Thanks for your reply. I couldn't find any errors in metron alerts UI >>>> log . I clicked the search and changed the date range too. Still no >>>> records. Do we have to run metron rest in dev profile? >>>> >>>> On Mon, Apr 8, 2019, 7:50 PM Michael Miklavcic < >>>> michael.miklav...@gmail.com> wrote: >>>> >>>>> If you see them in the dashboard you should be able to see them in the >>>>> alerts UI. Any errors in either the alerts UI or REST logs? Also, the new >>>>> default behavior is that the UI doesn't initiate a search at login, it's >>>>> up >>>>> to the user to click search. >>>>> >>>>> On Mon, Apr 8, 2019, 6:38 AM Hema malini <nhemamalin...@gmail.com> >>>>> wrote: >>>>> >>>>>> After recreating the index, now we are able to visualize the data in >>>>>> kibana metron dashboard. How we can pass alerts to metron alerts UI. >>>>>> Currently there is no data in alerts UI. How.to configure the logs as >>>>>> alerts >>>>>> >>>>>> On Sat, Apr 6, 2019, 9:21 PM Hema malini <nhemamalin...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Sorry for the typo. Can you please help with the required >>>>>>> configuration. >>>>>>> >>>>>>> On Sat, Apr 6, 2019, 5:39 PM Hema malini <nhemamalin...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Are we missing any configuration? Initially elastic search was >>>>>>>> down. We figured out the issue and fixed it .Now elastic search is up >>>>>>>> . We >>>>>>>> restarted metron indexing but still those indices not created. So we >>>>>>>> created it manually.Do we have to change any parser configuration . How >>>>>>>> logs will flow into metron alerts dashboard and kibana dashboard..what >>>>>>>> is >>>>>>>> the required congratulation >>>>>>>> >>>>>>>> On Fri, Apr 5, 2019, 11:52 PM Hema malini <nhemamalin...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Sample messages flown in indexing topic >>>>>>>>> {"msg":"'snort test alert'","parallelenricher.splitter.end.ts":" >>>>>>>>> 1554384505264","sig_rev":"0","ip_dst_port":"50183","ethsrc": >>>>>>>>> "08:00:27:E8:B0:7A","threat.triage.rules.0.comment":null," >>>>>>>>> tcpseq":"0x8DF34F4B","threat.triage.score":10.0,"dgmlen":" >>>>>>>>> 52","adapter.hostfromjsonlistadapter.end. >>>>>>>>> ts":"1554384503452","adapter.geoadapter.begin.ts":" >>>>>>>>> 1554384503452","tcpwindow":"0x1F5","parallelenricher. >>>>>>>>> splitter.begin.ts":"1554384505264","threat.triage. >>>>>>>>> rules.0.score":"10","tcpack":"0x836687BD","protocol":"TCP"," >>>>>>>>> ip_dst_addr":"192.168.66.1","original_string":"01\/11\/17-20:53:16.104984 >>>>>>>>> ,1,999158,0,\"'snort test alert'\",TCP,192.168.66.121, >>>>>>>>> 8080,192.168.66.1,50183,08:00:27:E8:B0:7A,0A:00:27:00:00:00, >>>>>>>>> 0x42,***A****,0x8DF34F4B,0x836687BD,,0x1F5,64,0,62040, >>>>>>>>> 52,53248,,,,","parallelenricher.enrich.end. >>>>>>>>> ts":"1554384505342","threat.triage.rules.0.reason":null," >>>>>>>>> tos":"0","adapter.hostfromjsonlistadapter.begin. >>>>>>>>> ts":"1554384503452","id":"62040","ip_src_addr":"192.168. >>>>>>>>> 66.121","timestamp":1484148196104,"ethdst":"0A:00:27:00:00:00"," >>>>>>>>> threat.triage.rules.0.name":null,"is_alert": >>>>>>>>> "true","parallelenricher.enrich.begin.ts":" >>>>>>>>> 1554384505264","ttl":"64","source.type":"snort","adapter. >>>>>>>>> geoadapter.end.ts":"1554384503453","ethlen":"0x42" >>>>>>>>> ,"iplen":"53248","adapter.threatinteladapter.begin.ts":" >>>>>>>>> 1554384505264","ip_src_port":"8080","tcpflags":"***A****"," >>>>>>>>> guid":"2f6f3f3c-7739-47fe-aa04-3c62425fbcbf","sig_id":" >>>>>>>>> 999158","sig_generator":"1"} >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Apr 5, 2019, 11:43 PM Hema malini <nhemamalin...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Yes I am getting messages >>>>>>>>>> >>>>>>>>>> On Fri, Apr 5, 2019, 11:17 PM Michael Miklavcic < >>>>>>>>>> michael.miklav...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Do you get 10 records output to the CLI when you run the >>>>>>>>>>> following? >>>>>>>>>>> >>>>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh >>>>>>>>>>> --zookeeper $ZOOKEEPER --topic indexing --from-beginning >>>>>>>>>>> --max-messages 10 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Apr 5, 2019 at 11:38 AM Hema malini < >>>>>>>>>>> nhemamalin...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> We verified it in Storm ui and in Storm topology logs >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Apr 5, 2019, 10:53 PM Michael Miklavcic < >>>>>>>>>>>> michael.miklav...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> How did you validate the logs are making it to the indexing >>>>>>>>>>>>> topology? >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Apr 5, 2019 at 8:12 AM Hema malini < >>>>>>>>>>>>> nhemamalin...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> We have installed Metron 0.7.1 in centos 7 using >>>>>>>>>>>>>> Amabari.Using Nifi we sent the sample snort logs copied from >>>>>>>>>>>>>> metron git >>>>>>>>>>>>>> repo to snort kafka topic.We did the same for bro topic.Logs are >>>>>>>>>>>>>> getting >>>>>>>>>>>>>> parsed and reached indexing topology . Elastic search indices >>>>>>>>>>>>>> are not >>>>>>>>>>>>>> getting created though we gave elastic search template install >>>>>>>>>>>>>> from ambari. >>>>>>>>>>>>>> So manually created the elastic search index using template >>>>>>>>>>>>>> available in metron repo. Though elastic search index is present >>>>>>>>>>>>>> , data >>>>>>>>>>>>>> from indexing toplogy neither reached elastic search nor hdfs >>>>>>>>>>>>>> path .There >>>>>>>>>>>>>> are no errors in storm toplogy logs.We could see the sample log >>>>>>>>>>>>>> in Metron >>>>>>>>>>>>>> management ui. How we can send the logs to alerts ui and kibana >>>>>>>>>>>>>> dashboard. >>>>>>>>>>>>>> In kibana dashboard we could see two dashboards - >>>>>>>>>>>>>> Metron-Dashboard,Metron-Error-Dashboard created but with no data. >>>>>>>>>>>>>> Elasticsearch health is yellow and we are able to insert data >>>>>>>>>>>>>> via rest >>>>>>>>>>>>>> call. Any documentation on sending the smaple snort logs to >>>>>>>>>>>>>> metron alerts >>>>>>>>>>>>>> ui will be helpful . Any configuration from metron management ui >>>>>>>>>>>>>> is >>>>>>>>>>>>>> required to pass it to alerts –ui >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks and Regards >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hema >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>
