MySQL should not have an influence on Kibana

On Fri, Feb 9, 2018 at 8:45 AM, zeo...@gmail.com <zeo...@gmail.com> wrote:

> Related:  The documentation
> <https://github.com/apache/metron/tree/master/metron-platform/metron-indexing#indexing-architecture>
> should probably be updated to account for METRON-1302.
>
> On Fri, Feb 9, 2018 at 8:18 AM R K Sharma <rksu...@gmail.com> wrote:
>
>> Thanks Michael... I can see similar information for my setup as well.
>> What I could notice is that MySQL service is not running on my machine
>> Which I think could be an issue for kibana. What's your view ?
>>
>> Regards
>> RK Sharma
>>
>> On Thu, Feb 8, 2018 at 9:11 PM, Michael Miklavcic <
>> michael.miklav...@gmail.com> wrote:
>>
>>> We now have 2 topologies for indexing - random access and batch. Double
>>> check that both are currently running - our full dev environment is pretty
>>> full with resources currently.
>>> random_access_indexing
>>> batch_indexing
>>>
>>> random_access_indexing is responsible for getting data into
>>> Elasticsearch. You can also check ES has indexes by going into Ambari ->
>>> Elasticsearch -> Quick Links -> Elasticsearch Indexes. You should see
>>> something like the following:
>>>
>>> health status index                     uuid                   pri rep 
>>> docs.count docs.deleted store.size pri.store.size
>>> yellow open   .kibana                   qbpdYf_RTMa_Rd2dB9q7oA   1   1      
>>>    44            0      120kb          120kb
>>> yellow open   bro_index_2018.02.06.22   -FiQxEGEQtSec0sC4oGAFA   5   1      
>>>  7990            0     12.8mb         12.8mb
>>> yellow open   bro_index_2018.02.06.23   AS4DHjrBQNyFrzDOxpGFeQ   5   1      
>>>  8100            0     12.7mb         12.7mb
>>> yellow open   snort_index_2018.02.06.20 Sxg-JGI3SAeXdg-V11BNkg   5   1      
>>>  7530            0     11.9mb         11.9mb
>>> yellow open   bro_index_2018.02.06.18   U1RTmFnpTCCDAicwWxc7Mg   5   1      
>>>  4640            0        8mb            8mb
>>>
>>> ...
>>>
>>>
>>>
>>>
>>> On Thu, Feb 8, 2018 at 3:19 AM, R K Sharma <rksu...@gmail.com> wrote:
>>>
>>>> Thanks Ryan...I see some data fro Snort & Bro sensors. Another  problem
>>>> which I have is that there is no information from Kibana dashboard.Do I
>>>> need to deploy some additional component to populate kibana ?
>>>>
>>>> Regards
>>>> RK Sharma
>>>>
>>>> On Wed, Feb 7, 2018 at 3:38 PM, Ryan Merriman <merrim...@gmail.com>
>>>> wrote:
>>>>
>>>>> I think you need to go one level deeper, those are directories.  Here
>>>>> is what I see in my dev environment:
>>>>>
>>>>> [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed
>>>>> Found 2 items
>>>>> drwxrwxr-x   - storm hadoop          0 2018-02-07 01:20
>>>>> /apps/metron/indexing/indexed/bro
>>>>> drwxrwxr-x   - storm hadoop          0 2018-02-07 01:20
>>>>> /apps/metron/indexing/indexed/snort
>>>>>
>>>>> [root@node1 ~]# hdfs dfs -ls /apps/metron/indexing/indexed/bro
>>>>> Found 1 items
>>>>> -rw-r--r--   1 storm hadoop   12842043 2018-02-07 01:20
>>>>> /apps/metron/indexing/indexed/bro/enrichment-hdfsIndexingBolt-3-0-
>>>>> 1517966421778.json
>>>>>
>>>>> On Wed, Feb 7, 2018 at 3:58 AM, R K Sharma <rksu...@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>       I have deployed Full Development VM on Virtual Box and all
>>>>>> services including metron, kafka, storm etc. are started. However, when I
>>>>>> check if there is some data written into HDFS (
>>>>>> /apps/metron/indexing/indexed/yaf|bro|snort ) for any data sources,
>>>>>> I don't see any data. Hereby below is output.
>>>>>>
>>>>>>
>>>>>>     [vagrant@node1 bin]$ hdfs dfs -ls /apps/metron/indexing/indexed/
>>>>>> Found 3 items
>>>>>> drwxrwxr-x   - storm hadoop          0 2018-02-06 13:03
>>>>>> /apps/metron/indexing/in
>>>>>>                                                                  
>>>>>> dexed/bro
>>>>>> drwxrwxr-x   - storm hadoop          0 2018-01-31 13:35
>>>>>> /apps/metron/indexing/in
>>>>>>                                                                  
>>>>>> dexed/error
>>>>>> drwxrwxr-x   - storm hadoop          0 2018-02-07 04:53
>>>>>> /apps/metron/indexing/in
>>>>>>                                                                  
>>>>>> dexed/snort
>>>>>>
>>>>>> On other-hand, I can see sensors (Snort & Bro) started on
>>>>>> http://node1:4200 and is having some throughput, although very low.
>>>>>> Hereby below is sensor status.
>>>>>>
>>>>>>
>>>>>> GrokWebSphere Stopped - -
>>>>>> jsonMap JSONMap Stopped - -
>>>>>> squid Grok Stopped - -
>>>>>> snort Snort Running 3.862s 1.89kb/s
>>>>>> asa Asa Stopped - -
>>>>>> bro Bro Running 4.25s 1.94kb/s
>>>>>> yaf Grok Running 0s 0kb/s
>>>>>> Can anybody guide me what should I check to ensure sensors produce
>>>>>> data and HDFS should be populated with this data ?
>>>>>>
>>>>>> Thanks & Regards
>>>>>> RK Sharma
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> --
>
> Jon
>

Reply via email to