MySQL should not have an influence on Kibana On Fri, Feb 9, 2018 at 8:45 AM, [email protected] <[email protected]> wrote:
> Related: The documentation > <https://github.com/apache/metron/tree/master/metron-platform/metron-indexing#indexing-architecture> > should probably be updated to account for METRON-1302. > > On Fri, Feb 9, 2018 at 8:18 AM R K Sharma <[email protected]> wrote: > >> Thanks Michael... I can see similar information for my setup as well. >> What I could notice is that MySQL service is not running on my machine >> Which I think could be an issue for kibana. What's your view ? >> >> Regards >> RK Sharma >> >> On Thu, Feb 8, 2018 at 9:11 PM, Michael Miklavcic < >> [email protected]> wrote: >> >>> We now have 2 topologies for indexing - random access and batch. Double >>> check that both are currently running - our full dev environment is pretty >>> full with resources currently. >>> random_access_indexing >>> batch_indexing >>> >>> random_access_indexing is responsible for getting data into >>> Elasticsearch. You can also check ES has indexes by going into Ambari -> >>> Elasticsearch -> Quick Links -> Elasticsearch Indexes. You should see >>> something like the following: >>> >>> health status index uuid pri rep >>> docs.count docs.deleted store.size pri.store.size >>> yellow open .kibana qbpdYf_RTMa_Rd2dB9q7oA 1 1 >>> 44 0 120kb 120kb >>> yellow open bro_index_2018.02.06.22 -FiQxEGEQtSec0sC4oGAFA 5 1 >>> 7990 0 12.8mb 12.8mb >>> yellow open bro_index_2018.02.06.23 AS4DHjrBQNyFrzDOxpGFeQ 5 1 >>> 8100 0 12.7mb 12.7mb >>> yellow open snort_index_2018.02.06.20 Sxg-JGI3SAeXdg-V11BNkg 5 1 >>> 7530 0 11.9mb 11.9mb >>> yellow open bro_index_2018.02.06.18 U1RTmFnpTCCDAicwWxc7Mg 5 1 >>> 4640 0 8mb 8mb >>> >>> ... >>> >>> >>> >>> >>> On Thu, Feb 8, 2018 at 3:19 AM, R K Sharma <[email protected]> wrote: >>> >>>> Thanks Ryan...I see some data fro Snort & Bro sensors. Another problem >>>> which I have is that there is no information from Kibana dashboard.Do I >>>> need to deploy some additional component to populate kibana ? >>>> >>>> Regards >>>> RK Sharma >>>> >>>> On Wed, Feb 7, 2018 at 3:38 PM, Ryan Merriman <[email protected]> >>>> wrote: >>>> >>>>> I think you need to go one level deeper, those are directories. Here >>>>> is what I see in my dev environment: >>>>> >>>>> [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed >>>>> Found 2 items >>>>> drwxrwxr-x - storm hadoop 0 2018-02-07 01:20 >>>>> /apps/metron/indexing/indexed/bro >>>>> drwxrwxr-x - storm hadoop 0 2018-02-07 01:20 >>>>> /apps/metron/indexing/indexed/snort >>>>> >>>>> [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed/bro >>>>> Found 1 items >>>>> -rw-r--r-- 1 storm hadoop 12842043 2018-02-07 01:20 >>>>> /apps/metron/indexing/indexed/bro/enrichment-hdfsIndexingBolt-3-0- >>>>> 1517966421778.json >>>>> >>>>> On Wed, Feb 7, 2018 at 3:58 AM, R K Sharma <[email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> I have deployed Full Development VM on Virtual Box and all >>>>>> services including metron, kafka, storm etc. are started. However, when I >>>>>> check if there is some data written into HDFS ( >>>>>> /apps/metron/indexing/indexed/yaf|bro|snort ) for any data sources, >>>>>> I don't see any data. Hereby below is output. >>>>>> >>>>>> >>>>>> [vagrant@node1 bin]$ hdfs dfs -ls /apps/metron/indexing/indexed/ >>>>>> Found 3 items >>>>>> drwxrwxr-x - storm hadoop 0 2018-02-06 13:03 >>>>>> /apps/metron/indexing/in >>>>>> >>>>>> dexed/bro >>>>>> drwxrwxr-x - storm hadoop 0 2018-01-31 13:35 >>>>>> /apps/metron/indexing/in >>>>>> >>>>>> dexed/error >>>>>> drwxrwxr-x - storm hadoop 0 2018-02-07 04:53 >>>>>> /apps/metron/indexing/in >>>>>> >>>>>> dexed/snort >>>>>> >>>>>> On other-hand, I can see sensors (Snort & Bro) started on >>>>>> http://node1:4200 and is having some throughput, although very low. >>>>>> Hereby below is sensor status. >>>>>> >>>>>> >>>>>> GrokWebSphere Stopped - - >>>>>> jsonMap JSONMap Stopped - - >>>>>> squid Grok Stopped - - >>>>>> snort Snort Running 3.862s 1.89kb/s >>>>>> asa Asa Stopped - - >>>>>> bro Bro Running 4.25s 1.94kb/s >>>>>> yaf Grok Running 0s 0kb/s >>>>>> Can anybody guide me what should I check to ensure sensors produce >>>>>> data and HDFS should be populated with this data ? >>>>>> >>>>>> Thanks & Regards >>>>>> RK Sharma >>>>>> >>>>>> >>>>> >>>> >>> >> -- > > Jon >
