[ https://issues.apache.org/jira/browse/METRON-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080560#comment-16080560 ]
ASF GitHub Bot commented on METRON-1026: ---------------------------------------- GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/643 METRON-1026: threatintel_taxii_load.sh throws exception ## Contributor Comments The delegation wrapping done to enable Stellar transformations on data as it is imported broke the taxii loader. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` #### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cestella/incubator-metron taxii_error Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/643.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #643 ---- ---- > threatintel_taxii_load.sh throws exception > ------------------------------------------ > > Key: METRON-1026 > URL: https://issues.apache.org/jira/browse/METRON-1026 > Project: Metron > Issue Type: Bug > Affects Versions: 0.3.1, 0.3.2, 0.4.0 > Reporter: Vladimir > > For version 0.3.1 and later taxii loader does not work. > Here is details: > {noformat} > # /usr/metron/0.3.1/bin/threatintel_taxii_load.sh -c > /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json -e > /usr/local/opentaxii/etc/connections.conf/extractor.json -p 10000 > 17/07/10 07:35:42 WARN extractor.TransformFilterExtractorDecorator: Unable to > setup zookeeper client - zk_quorum url not provided. **This will limit some > Stellar functionality** > Exception in thread "main" java.lang.IllegalStateException: Extractor must be > a STIX Extractor > at > org.apache.metron.dataloads.nonbulk.taxii.TaxiiLoader.main(TaxiiLoader.java:202) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.hadoop.util.RunJar.run(RunJar.java:233) > at org.apache.hadoop.util.RunJar.main(RunJar.java:148) > {noformat} > Configs: > {noformat} > # cat /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json > { > "endpoint" : "http://10.10.110.23:9000/services/discovery" > ,"port" : "9000" > ,"type" : "DISCOVER" > ,"collection" : "guest.phishtank_com" > ,"table" : "threatintel" > ,"columnFamily" : "t" > ,"allowedIndicatorTypes" : [ "domainname:FQDN", "address:IPV_4_ADDR" ] > } > {noformat} > and > {noformat} > # cat /usr/local/opentaxii/etc/connections.conf/extractor.json > { > "config": { > "columns": { > "domain": 0 > }, > "indicator_column": "domain", > "type" : "malicious_domain", > "separator" : "," > }, > "extractor" : "STIX" > } > {noformat} > The same exception with *"zk_quorum"* parameter (but without "Unable to setup > zookeeper client" warning). > Exception is thrown due to false in condition: > {code} > if(e instanceof StixExtractor) { > {code} > in file *TaxiiLoader.java*. > I changed exception message to: > {code} > throw new IllegalStateException("Extractor must be a STIX Extractor" + " - " > + e.getClass().getName()); > {code} > and got: > {noformat} > Extractor must be a STIX Extractor - > org.apache.metron.dataloads.extractor.TransformFilterExtractorDecorator > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)