Justin Leet created METRON-527:
----------------------------------
Summary: Connection pool shut down seen when running
threatintel_taxii_load.sh script
Key: METRON-527
URL: https://issues.apache.org/jira/browse/METRON-527
Project: Metron
Issue Type: Bug
Reporter: Justin Leet
Assignee: Justin Leet
Discovered by [~anandsubbu]:
I have setup opentaxii server on a 12-node metron cluster. I was able to scan
and pull records into the taxi server.
{code}
[root@metron-test2-8 ~]# service opentaxii status
Checking opentaxii... Running
guest.phishtank_com 5891
guest.Abuse_ch 49
guest.CyberCrime_Tracker 0
guest.EmergingThreats_rules 0
guest.Lehigh_edu 2025
guest.MalwareDomainList_Hostlist 50
guest.blutmagie_de_torExits 14897
guest.dataForLast_7daysOnly 23210
guest.dshield_BlockList 0
{code}
When I ran the threatintel_taxii_load.sh script, it is able to push some of the
records into hbase 'threatintel' table, but fails with a "Connection pool shut
down" error shortly after. This was observed for all the subscriptions that I
attempted.
Here is the command that was used:
{code}
[root@metron-test2-8 ~]# /usr/metron/0.2.0BETA/bin//threatintel_taxii_load.sh
-b "2015-01-01 00:00:00" -c ~/connection_config.json -e ~/extractor.json -p
300000
{code}
The error message is pasted below.
{code}
16/08/30 09:24:14 ERROR taxii.TaxiiHandler: Connection pool shut down
java.lang.IllegalStateException: Connection pool shut down
at
org.apache.metron.httpcore.dataload.util.Asserts.check(Asserts.java:34)
at
org.apache.metron.httpcore.dataload.pool.AbstractConnPool.lease(AbstractConnPool.java:169)
at
org.apache.metron.httpcore.dataload.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:217)
at
org.apache.metron.httpcore.dataload.impl.execchain.MainClientExec.execute(MainClientExec.java:158)
at
org.apache.metron.httpcore.dataload.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at
org.apache.metron.httpcore.dataload.impl.execchain.RetryExec.execute(RetryExec.java:85)
at
org.apache.metron.httpcore.dataload.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at
org.apache.metron.httpcore.dataload.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at
org.apache.metron.httpcore.dataload.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at
org.mitre.taxii.client.HttpClient.callTaxiiService(HttpClient.java:297)
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:336)
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:242)
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.run(TaxiiHandler.java:171)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Exception in thread "Timer-0" java.lang.RuntimeException: Unable to make request
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.run(TaxiiHandler.java:214)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: java.lang.IllegalStateException: Connection pool shut down
at
org.apache.metron.httpcore.dataload.util.Asserts.check(Asserts.java:34)
at
org.apache.metron.httpcore.dataload.pool.AbstractConnPool.lease(AbstractConnPool.java:169)
at
org.apache.metron.httpcore.dataload.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:217)
at
org.apache.metron.httpcore.dataload.impl.execchain.MainClientExec.execute(MainClientExec.java:158)
at
org.apache.metron.httpcore.dataload.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at
org.apache.metron.httpcore.dataload.impl.execchain.RetryExec.execute(RetryExec.java:85)
at
org.apache.metron.httpcore.dataload.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at
org.apache.metron.httpcore.dataload.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at
org.apache.metron.httpcore.dataload.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at
org.mitre.taxii.client.HttpClient.callTaxiiService(HttpClient.java:297)
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:336)
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:242)
at
org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.run(TaxiiHandler.java:171)
... 2 more
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
