[
https://issues.apache.org/jira/browse/METRON-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710739#comment-16710739
]
ASF GitHub Bot commented on METRON-1916:
----------------------------------------
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1288
It was when I was working on #1289 . After changing the POM, I found that
multiple versions of httpclient were getting pulled in, which broke all the
integration tests because a Stellar executor could not load. It would throw
the exception that I put in the JIRA.
This is one specific scenario, but I can imagine that classpath issues like
this are going to crop-up in different environments. It seems like we should
have a way to handle this.
> Stellar Classpath Function Resolver Should Handle ClassNotFoundException
> ------------------------------------------------------------------------
>
> Key: METRON-1916
> URL: https://issues.apache.org/jira/browse/METRON-1916
> Project: Metron
> Issue Type: Improvement
> Reporter: Nick Allen
> Assignee: Nick Allen
> Priority: Major
>
> Stellar can be executed in an environment where not all of the dependencies
> are available for each of the functions provided in Stellar Common. When this
> happens, the classpath resolver blows-up even if you do not use the function
> that is missing dependencies.
> For example this exception is thrown when not all of the dependencies are
> available for the new REST function. I was only attempting to use
> STATS_INIT, not the REST function.
> {code:java}
> 2018-11-30 15:47:00 DEBUG BaseFunctionResolver:165 - Resolving Stellar
> function in class RestFunctions.RestGet
> 2018-11-30 15:47:00 ERROR DefaultProfileBuilder:302 - Bad 'init' expression:
> error='Unable to parse: STATS_INIT() due to:
> org/apache/http/conn/HttpClientConnectionManager', expr='STATS_INIT()',
> profile='profile-with-stats', entity='global',
> variables-available='[adapter.threatinteladapter.end.ts, bro_timestamp,
> ip_dst_port, enrichmentsplitterbolt.splitter.end.ts,
> enrichmentsplitterbolt.splitter.begin.ts,
> adapter.hostfromjsonlistadapter.end.ts, adapter.geoadapter.begin.ts, uid,
> trans_depth, protocol, original_string, ip_dst_addr,
> threatinteljoinbolt.joiner.ts, host, enrichmentjoinbolt.joiner.ts,
> adapter.hostfromjsonlistadapter.begin.ts,
> threatintelsplitterbolt.splitter.begin.ts, ip_src_addr, user_agent,
> timestamp, method, request_body_len, uri, tags, source.type,
> adapter.geoadapter.end.ts, referrer, threatintelsplitterbolt.splitter.end.ts,
> adapter.threatinteladapter.begin.ts, ip_src_port, guid, response_body_len]'
> org.apache.metron.stellar.dsl.ParseException: Unable to parse: STATS_INIT()
> due to: org/apache/http/conn/HttpClientConnectionManager
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:166)
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154)
> at
> org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.execute(DefaultStellarStatefulExecutor.java:160)
> at
> org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.assign(DefaultStellarStatefulExecutor.java:95)
> at
> org.apache.metron.profiler.DefaultProfileBuilder.assign(DefaultProfileBuilder.java:291)
> at
> org.apache.metron.profiler.DefaultProfileBuilder.apply(DefaultProfileBuilder.java:136)
> at
> org.apache.metron.profiler.DefaultMessageDistributor.distribute(DefaultMessageDistributor.java:177)
> at
> org.apache.metron.profiler.storm.ProfileBuilderBolt.handleMessage(ProfileBuilderBolt.java:411)
> at
> org.apache.metron.profiler.storm.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:337)
> at
> org.apache.storm.topology.WindowedBoltExecutor$1.onActivation(WindowedBoltExecutor.java:307)
> at org.apache.storm.windowing.WindowManager.onTrigger(WindowManager.java:145)
> at
> org.apache.storm.windowing.WatermarkTimeTriggerPolicy.handleWaterMarkEvent(WatermarkTimeTriggerPolicy.java:80)
> at
> org.apache.storm.windowing.WatermarkTimeTriggerPolicy.track(WatermarkTimeTriggerPolicy.java:48)
> at org.apache.storm.windowing.WindowManager.track(WindowManager.java:177)
> at org.apache.storm.windowing.WindowManager.add(WindowManager.java:110)
> at
> org.apache.storm.windowing.WaterMarkEventGenerator.run(WaterMarkEventGenerator.java:80)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/http/conn/HttpClientConnectionManager
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.getConstructor(Class.java:1825)
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.createFunction(BaseFunctionResolver.java:240)
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.resolveFunction(BaseFunctionResolver.java:194)
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.resolveFunctions(BaseFunctionResolver.java:166)
> at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:109)
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:149)
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:49)
> at
> org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:688)
> at
> org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:656)
> at
> org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259)
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151)
> ... 21 more{code}
> When this happens the Classpath Function Resolver should allow me to use
> whichever functions can be loaded correctly.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
