Thanks Michael... I can see similar information for my setup as well. What I could notice is that MySQL service is not running on my machine Which I think could be an issue for kibana. What's your view ?
Regards RK Sharma On Thu, Feb 8, 2018 at 9:11 PM, Michael Miklavcic < [email protected]> wrote: > We now have 2 topologies for indexing - random access and batch. Double > check that both are currently running - our full dev environment is pretty > full with resources currently. > random_access_indexing > batch_indexing > > random_access_indexing is responsible for getting data into Elasticsearch. > You can also check ES has indexes by going into Ambari -> Elasticsearch -> > Quick Links -> Elasticsearch Indexes. You should see something like the > following: > > health status index uuid pri rep > docs.count docs.deleted store.size pri.store.size > yellow open .kibana qbpdYf_RTMa_Rd2dB9q7oA 1 1 > 44 0 120kb 120kb > yellow open bro_index_2018.02.06.22 -FiQxEGEQtSec0sC4oGAFA 5 1 > 7990 0 12.8mb 12.8mb > yellow open bro_index_2018.02.06.23 AS4DHjrBQNyFrzDOxpGFeQ 5 1 > 8100 0 12.7mb 12.7mb > yellow open snort_index_2018.02.06.20 Sxg-JGI3SAeXdg-V11BNkg 5 1 > 7530 0 11.9mb 11.9mb > yellow open bro_index_2018.02.06.18 U1RTmFnpTCCDAicwWxc7Mg 5 1 > 4640 0 8mb 8mb > > ... > > > > > On Thu, Feb 8, 2018 at 3:19 AM, R K Sharma <[email protected]> wrote: > >> Thanks Ryan...I see some data fro Snort & Bro sensors. Another problem >> which I have is that there is no information from Kibana dashboard.Do I >> need to deploy some additional component to populate kibana ? >> >> Regards >> RK Sharma >> >> On Wed, Feb 7, 2018 at 3:38 PM, Ryan Merriman <[email protected]> >> wrote: >> >>> I think you need to go one level deeper, those are directories. Here is >>> what I see in my dev environment: >>> >>> [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed >>> Found 2 items >>> drwxrwxr-x - storm hadoop 0 2018-02-07 01:20 >>> /apps/metron/indexing/indexed/bro >>> drwxrwxr-x - storm hadoop 0 2018-02-07 01:20 >>> /apps/metron/indexing/indexed/snort >>> >>> [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed/bro >>> Found 1 items >>> -rw-r--r-- 1 storm hadoop 12842043 2018-02-07 01:20 >>> /apps/metron/indexing/indexed/bro/enrichment-hdfsIndexingBol >>> t-3-0-1517966421778.json >>> >>> On Wed, Feb 7, 2018 at 3:58 AM, R K Sharma <[email protected]> wrote: >>> >>>> Hi, >>>> I have deployed Full Development VM on Virtual Box and all >>>> services including metron, kafka, storm etc. are started. However, when I >>>> check if there is some data written into HDFS ( >>>> /apps/metron/indexing/indexed/yaf|bro|snort ) for any data sources, I >>>> don't see any data. Hereby below is output. >>>> >>>> >>>> [vagrant@node1 bin]$ hdfs dfs -ls /apps/metron/indexing/indexed/ >>>> Found 3 items >>>> drwxrwxr-x - storm hadoop 0 2018-02-06 13:03 >>>> /apps/metron/indexing/in >>>> dexed/bro >>>> drwxrwxr-x - storm hadoop 0 2018-01-31 13:35 >>>> /apps/metron/indexing/in >>>> >>>> dexed/error >>>> drwxrwxr-x - storm hadoop 0 2018-02-07 04:53 >>>> /apps/metron/indexing/in >>>> >>>> dexed/snort >>>> >>>> On other-hand, I can see sensors (Snort & Bro) started on >>>> http://node1:4200 and is having some throughput, although very low. >>>> Hereby below is sensor status. >>>> >>>> >>>> GrokWebSphere Stopped - - >>>> jsonMap JSONMap Stopped - - >>>> squid Grok Stopped - - >>>> snort Snort Running 3.862s 1.89kb/s >>>> asa Asa Stopped - - >>>> bro Bro Running 4.25s 1.94kb/s >>>> yaf Grok Running 0s 0kb/s >>>> Can anybody guide me what should I check to ensure sensors produce data >>>> and HDFS should be populated with this data ? >>>> >>>> Thanks & Regards >>>> RK Sharma >>>> >>>> >>> >> >
