[
https://issues.apache.org/jira/browse/METRON-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120872#comment-16120872
]
ASF GitHub Bot commented on METRON-1026:
----------------------------------------
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/643
Presumptions:
* Fulldev has opentaxii installed with the `guest.phishtank_com` collection
configured
Test:
* Ensure that opentaxii is running by running `service opentaxii status`
* Ensure that the collection is sync'd by running `service opentaxii sync
guest.phishtank_com 2017-08-01`
* Open `~/taxii.json` and input the following:
```
{
"endpoint" : "http://localhost:9000/services/discovery";
,"type" : "DISCOVER"
,"collection" : "guest.phishtank_com"
,"table" : "threatintel"
,"columnFamily" : "t"
,"allowedIndicatorTypes" : [ ]
}
```
* Create a file `~/extractor.json` with the following:
```
{
"extractor" : "STIX",
"config" : {}
}
```
* Run the taxii loader via `/usr/metron/0.4.1/bin/threatintel_taxii_load.sh
-e ./extractor.json -c ./taxii.json`
* Ensure it functions and data flows as usual (or not, depending on what
data is in hailataxii). Prior to this PR, you'd have seen an exception.
One thing to note, most of the recent hailataxii data are URIs which isn't
something that we support. I patched this branch with support for URIs and
verified data was flowing into HBase. That URI support is done via #689
> threatintel_taxii_load.sh throws exception
> ------------------------------------------
>
> Key: METRON-1026
> URL: https://issues.apache.org/jira/browse/METRON-1026
> Project: Metron
> Issue Type: Bug
> Affects Versions: 0.3.1, 0.3.2, 0.4.0
> Reporter: Vladimir
>
> For version 0.3.1 and later taxii loader does not work.
> Here is details:
> {noformat}
> # /usr/metron/0.3.1/bin/threatintel_taxii_load.sh -c
> /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json -e
> /usr/local/opentaxii/etc/connections.conf/extractor.json -p 10000
> 17/07/10 07:35:42 WARN extractor.TransformFilterExtractorDecorator: Unable to
> setup zookeeper client - zk_quorum url not provided. **This will limit some
> Stellar functionality**
> Exception in thread "main" java.lang.IllegalStateException: Extractor must be
> a STIX Extractor
> at
> org.apache.metron.dataloads.nonbulk.taxii.TaxiiLoader.main(TaxiiLoader.java:202)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
> {noformat}
> Configs:
> {noformat}
> # cat /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json
> {
> "endpoint" : "http://10.10.110.23:9000/services/discovery";
> ,"port" : "9000"
> ,"type" : "DISCOVER"
> ,"collection" : "guest.phishtank_com"
> ,"table" : "threatintel"
> ,"columnFamily" : "t"
> ,"allowedIndicatorTypes" : [ "domainname:FQDN", "address:IPV_4_ADDR" ]
> }
> {noformat}
> and
> {noformat}
> # cat /usr/local/opentaxii/etc/connections.conf/extractor.json
> {
> "config": {
> "columns": {
> "domain": 0
> },
> "indicator_column": "domain",
> "type" : "malicious_domain",
> "separator" : ","
> },
> "extractor" : "STIX"
> }
> {noformat}
> The same exception with *"zk_quorum"* parameter (but without "Unable to setup
> zookeeper client" warning).
> Exception is thrown due to false in condition:
> {code}
> if(e instanceof StixExtractor) {
> {code}
> in file *TaxiiLoader.java*.
> I changed exception message to:
> {code}
> throw new IllegalStateException("Extractor must be a STIX Extractor" + " - "
> + e.getClass().getName());
> {code}
> and got:
> {noformat}
> Extractor must be a STIX Extractor -
> org.apache.metron.dataloads.extractor.TransformFilterExtractorDecorator
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
