Hi, I had similar issue it turned out to be the issue in STROM
No worker is assigned to togolgoy all you need is to add additional port in Ambari -> Storm -> Configs -> supervisor.slot.ports by assigning an additional port to the list https://community.hortonworks.com/questions/32499/no-workers-in-storm-for-squid-topology.html I had similar issue and finally got it fixed On Mon, Jan 15, 2018 at 8:45 AM, Gaurav Bapat <gauravb3...@gmail.com> wrote: > Storm UI > > On 15 January 2018 at 08:59, Gaurav Bapat <gauravb3...@gmail.com> wrote: > >> Hey Jon, >> >> I have Storm UI and the logs are coming from firewalls, servers, etc from >> other machines(HP ArcSight Logger). >> >> I have attached the NiFi screenshots, my logs are coming but there is >> some error with Kafka and I am having issues with configuring Kafka broker >> >> >> >> On 12 January 2018 at 18:14, zeo...@gmail.com <zeo...@gmail.com> wrote: >> >>> In Ambari under storm you can find the UI under quick links at the top. >>> That said, the issue seems to be upstream of Metron, in NiFi. That is >>> something I can't help with as much, but if you can share the listensyslog >>> processor config that would be a start. Also, share the config of the >>> thing that is sending syslog as well (are these local syslog, is that >>> machine aggregating syslog from other machines, etc.). Thanks, >>> >>> Jon >>> >>> On Fri, Jan 12, 2018, 01:00 Gaurav Bapat <gauravb3...@gmail.com> wrote: >>> >>>> I have created a Kafka topic "cef" but my Listen Syslogs is not getting >>>> logs in the processor. >>>> >>>> Also I checked using tcpdump -i and it is getting logs in my machine >>>> but ListenSyslogs is not getting the logs >>>> >>>> On 12 January 2018 at 11:13, Gaurav Bapat <gauravb3...@gmail.com> >>>> wrote: >>>> >>>>> [root@metron incubator-metron]# ./metron-deployment/scripts/pl >>>>> atform-info.sh >>>>> Metron 0.4.3 >>>>> -- >>>>> * master >>>>> -- >>>>> commit c559ed7e1838ec71344eae3d9e37771db2641635 >>>>> Author: cstella <ceste...@gmail.com> >>>>> Date: Tue Jan 9 15:28:47 2018 -0500 >>>>> >>>>> METRON-1379: Add an OBJECT_GET stellar function closes >>>>> apache/incubator-metron#880 >>>>> -- >>>>> metron-deployment/vagrant/full-dev-platform/Vagrantfile | 2 +- >>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>> -- >>>>> ansible 2.0.0.2 >>>>> config file = >>>>> configured module search path = Default w/o overrides >>>>> -- >>>>> Vagrant 1.9.6 >>>>> -- >>>>> Python 2.7.5 >>>>> -- >>>>> Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; >>>>> 2015-11-10T22:11:47+05:30) >>>>> Maven home: /opt/maven/current >>>>> Java version: 1.8.0_151, vendor: Oracle Corporation >>>>> Java home: /opt/jdk1.8.0_151/jre >>>>> Default locale: en_US, platform encoding: UTF-8 >>>>> OS name: "linux", version: "3.10.0-693.11.6.el7.x86_64", arch: >>>>> "amd64", family: "unix" >>>>> -- >>>>> Docker version 1.12.6, build ec8512b/1.12.6 >>>>> -- >>>>> node >>>>> v8.9.3 >>>>> -- >>>>> npm >>>>> 5.5.1 >>>>> -- >>>>> g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16) >>>>> Copyright (C) 2015 Free Software Foundation, Inc. >>>>> This is free software; see the source for copying conditions. There >>>>> is NO >>>>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR >>>>> PURPOSE. >>>>> >>>>> -- >>>>> Compiler is C++11 compliant >>>>> -- >>>>> Linux metron.com 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Jan 4 01:06:37 >>>>> UTC 2018 x86_64 x86_64 x86_64 GNU/Linux >>>>> -- >>>>> Total System Memory = 15773.3 MB >>>>> Processor Model: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz >>>>> Processor Speed: 3320.875 MHz >>>>> Processor Speed: 3307.191 MHz >>>>> Processor Speed: 3376.699 MHz >>>>> Processor Speed: 3338.917 MHz >>>>> Total Physical Processors: 4 >>>>> Total cores: 16 >>>>> Disk information: >>>>> /dev/mapper/centos-root 200G 22G 179G 11% / >>>>> /dev/sda1 2.0G 224M 1.8G 11% /boot >>>>> /dev/sda2 1022M 12K 1022M 1% /boot/efi >>>>> /dev/mapper/centos-home 247G 10G 237G 5% /home >>>>> This CPU appears to support virtualization >>>>> >>>>> On 12 January 2018 at 09:25, Gaurav Bapat <gauravb3...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hey Jon, >>>>>> >>>>>> Appreciate your timely reply. >>>>>> >>>>>> I gone through your answer but still I can't figure out how do I do >>>>>> parsing/indexing in Storm UI as I cant find any option for the same. >>>>>> >>>>>> Is there any other UI to do parsing/indexing? >>>>>> >>>>>> >>>>>> >>>>>> On 11 January 2018 at 21:22, zeo...@gmail.com <zeo...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> So, you created a new cef topic, and set up the appropriate parser >>>>>>> config for it (if not, this >>>>>>> <https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source> >>>>>>> may be helpful)? If so: >>>>>>> >>>>>>> Here are some basic troubleshooting steps: >>>>>>> 1. Validate that the logs are getting onto the kafka topic that you >>>>>>> are sending to. If they aren't there, the problem is upstream from >>>>>>> Metron. >>>>>>> 2. If they are getting onto the kafka topic they are being directly >>>>>>> sent to, check the indexing kafka topic for an enriched version of those >>>>>>> same logs. >>>>>>> 3. Do a binary search of the various components involved with >>>>>>> ingest. >>>>>>> a. If the logs are *not* on the indexing kafka topic, check the >>>>>>> enrichments topic for those logs. >>>>>>> b. If the logs are *not* on the enrichments topic, check the >>>>>>> parser storm topology. >>>>>>> c. If the logs are on the enrichments topic, but *not* >>>>>>> indexing, check the enrichments storm topology. >>>>>>> d. If the logs are on the indexing but *not* Kibana, check the >>>>>>> indexing storm topic. >>>>>>> e. If the logs are in on the indexing topic and indexing storm >>>>>>> topic is in good shape, check elasticsearch directly. >>>>>>> 4. You should have identified where the issue is at this point. >>>>>>> Report back here with what you observed, any relevant error messages, >>>>>>> etc. >>>>>>> >>>>>>> Side note: We should document a decision tree for troubleshooting >>>>>>> data ingest. It is fairly straightforward and makes me wonder if we >>>>>>> already have this somewhere and I'm not aware of it? It would also be a >>>>>>> good place to put pointers to some common errors. >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On Thu, Jan 11, 2018 at 1:44 AM Gaurav Bapat <gauravb3...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hello everyone, I have deployed Metron on a single node machine and >>>>>>>> I would like to know how do I get Syslogs from NiFi into Kibana >>>>>>>> dashboard? >>>>>>>> >>>>>>>> I have created a Kafka topic by the name "cef" and I can see that >>>>>>>> the topic exists in >>>>>>>> Metron Configuration but I am unable to connect it with Kibana >>>>>>>> >>>>>>>> Need Help!! >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Jon >>>>>>> >>>>>> >>>>>> >>>>> >>>> -- >>> >>> Jon >>> >> >> > -- With Regards Farrukh Naveed Anjum
