[ 
https://issues.apache.org/jira/browse/METRON-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569026#comment-15569026
 ] 

ASF GitHub Bot commented on METRON-484:
---------------------------------------

GitHub user nickwallen opened a pull request:

    https://github.com/apache/incubator-metron/pull/306

    METRON-484 Opentaxii service does not show count for subscribed services

    [METRON-484](https://issues.apache.org/jira/browse/METRON-484)
    
    This PR was a collaboration between myself and @nsinha17.
    
    There were recent changes in the Cabby Python module used as part of the 
deployed Opentaxii service.  These changes broke the service script which 
prevented it from listing the number of threat indicators for each collection.
    
    ### Testing
    
    Deployed Opentaxii on Quick-Dev.  Able to list counts by service and load 
threat data.  See transcript below.
    
    ```
    [root@node1 ~]# service opentaxii status
    Checking opentaxii...                             Running
    guest.phishtank_com                                0
    guest.Abuse_ch                                     0
    guest.CyberCrime_Tracker                           0
    guest.EmergingThreats_rules                        0
    guest.Lehigh_edu                                   0
    guest.MalwareDomainList_Hostlist                   0
    guest.blutmagie_de_torExits                        0
    guest.dataForLast_7daysOnly                        0
    guest.dshield_BlockList                            0
    
    [root@node1 ~]# service opentaxii sync guest.phishtank_com
    + /usr/local/opentaxii/opentaxii-venv/bin/taxii-proxy --poll-path 
http://hailataxii.com/taxii-data --poll-collection guest.phishtank_com 
--inbox-path http://localhost:9000/services/inbox --inbox-collection 
guest.phishtank_com --binding urn:stix.mitre.org:xml:1.1.1 --begin 2016-10-12 
--end 2016-10-13
    2016-10-12 15:19:28,701 INFO: Sending Poll_Request to 
http://hailataxii.com/taxii-data
    2016-10-12 15:19:28,704 INFO: Starting new HTTP connection (1): 
hailataxii.com
    2016-10-12 15:19:28,942 INFO: Response received for Poll_Request from 
http://hailataxii.com/taxii-data
    2016-10-12 15:19:29,046 INFO: Sending Inbox_Message to 
http://localhost:9000/services/inbox
    2016-10-12 15:19:29,048 INFO: Starting new HTTP connection (1): localhost
    ...
    2016-10-12 15:20:28,317 INFO: 1762 blocks polled
    2016-10-12 15:20:28,317 INFO: Content block successfully pushed
    + set +x
    2016-10-12 15:20:28,436 INFO: Polling using data binding: ALL
    2016-10-12 15:20:28,441 INFO: Sending Discovery_Request to 
http://localhost:9000/services/discovery
    2016-10-12 15:20:28,444 INFO: Starting new HTTP connection (1): localhost
    2016-10-12 15:20:28,450 INFO: Response received for Discovery_Request from 
http://localhost:9000/services/discovery
    2016-10-12 15:20:28,451 INFO: 3 services discovered
    2016-10-12 15:20:28,452 INFO: Sending Poll_Request to 
http://localhost:9000/services/poll
    2016-10-12 15:20:28,453 INFO: Starting new HTTP connection (1): localhost
    2016-10-12 15:20:28,462 INFO: Response received for Poll_Request from 
http://localhost:9000/services/poll
    2016-10-12 15:20:28,462 INFO: Content blocks count: 1762, is partial: False
    
    [root@node1 ~]# service opentaxii status
    Checking opentaxii...                             Running
    guest.phishtank_com                                1762
    guest.Abuse_ch                                     0
    guest.CyberCrime_Tracker                           0
    guest.EmergingThreats_rules                        0
    guest.Lehigh_edu                                   0
    guest.MalwareDomainList_Hostlist                   0
    guest.blutmagie_de_torExits                        0
    guest.dataForLast_7daysOnly                        0
    guest.dshield_BlockList                            0
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/nickwallen/incubator-metron METRON-484

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/306.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #306
    
----
commit ee65589eea36be441cd19709524a4ba015b937a8
Author: nsinha17 <nsi...@hortonworks.com>
Date:   2016-10-05T06:36:52Z

    METRON-484: Opentaxi service does not show count for subscribed services

commit 3e0b05e1858c4173fb3078fc36c4499421ba8fe4
Author: nsinha17 <nsi...@hortonworks.com>
Date:   2016-10-05T08:49:49Z

    METRON-484: Opentaxi service does not show count for subscribed services

commit d538fcdc2b29111d9949cb6e1cd9a6c5c1372527
Author: Nick Allen <n...@nickallen.org>
Date:   2016-10-12T15:01:04Z

    METRON-484 Allow use of http, https, or ftp and set bind variable 
appropriately

commit c982f68b6c02f3da997ce51a05bcc83aa0ad3f27
Author: Nick Allen <n...@nickallen.org>
Date:   2016-10-12T15:01:48Z

    Merge remote-tracking branch 'apache/master' into METRON-484

commit 95f9b473b5bd1961ecfa9bd7119382c436b55d55
Author: Nick Allen <n...@nickallen.org>
Date:   2016-10-12T15:02:49Z

    METRON-484 Allow use of http, https, or ftp and set bind variable 
appropriately

commit 0a0f2e5fb45435cc82f6b3792da94b49631dae2d
Author: Nick Allen <n...@nickallen.org>
Date:   2016-10-12T15:21:31Z

    METRON-484 Sync was not working because protocol was specified twice

----


> Opentaxi service does not show count for subscribed services  
> --------------------------------------------------------------
>
>                 Key: METRON-484
>                 URL: https://issues.apache.org/jira/browse/METRON-484
>             Project: Metron
>          Issue Type: Bug
>    Affects Versions: 0.2.1BETA
>         Environment: 12 node cluster running CentOS 7 on Openstack.
>            Reporter: Anand Subramanian
>
> *Steps Followed*
> a) Created an ansible playbook for 'opentaxii' role and installed on a 
> pre-deployed 12-node cluster with the following playbook definition:
> {code}
> - hosts: metron
>   become: true
>   roles:
>     - role: opentaxii
> {code}
> b) Followed the documentation at:
> https://github.com/apache/incubator-metron/tree/master/metron-deployment/roles/opentaxii#getting-started
> c) Ensure that the opentaxii service is installed on the 'metron' nodes viz. 
> 1, 3, 8, 10, 11 and 12.
> Observed on all the nodes that the 'status' command output is as follows:
> {code}
> [root@metron-test-1 ~]# service opentaxii status
> Checking opentaxii...                             Running
> Services not defined
> {code}
> *Workaround*
> In order to get around this issue, edit the file at 
> /usr/local/opentaxii/bin/collection-status.py and change it to look like the 
> below.
> {code}
> <snip>
> from cabby import create_client
> base_uri = 'http://localhost:9000'
> try:
>     # create a connection
>     client = 
> create_client(host='localhost',port=9000,discovery_path='/services/discovery')
>     # iterate through each defined collection
>     collections = 
> client.get_collections(uri='%s/services/collection'%base_uri)
>     for collection in collections:
>         # how many records in each collection?
>         count = client.get_content_count(collection_name=collection.name, 
> uri='%s/services/poll'%base_uri)
>         print "%-50s %-10d" % (collection.name, count.count)
> except:
>     print "Services not defined"
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to