mmiklavc commented on issue #1483: METRON-2217 Migrate current HBase client from HTableInterface to Table URL: https://github.com/apache/metron/pull/1483#issuecomment-525960987 I was getting classpath issues with my PR that now seem to be fixed with https://github.com/apache/metron/pull/1498. That PR should get merged tomorrow sometime, barring any discussion to the contrary. In the meantime, I've merged those changes into this PR for sake of testing. I ran through the major parts of the test plan one more time and everything works as expected. I did not repeat the failure recovery scenario. For reference, this another exception I ran into while running the basic parser test. Again, this issue appears to have been resolved now with the changes in pr 1498: ``` 2019-08-27 20:38:55.531 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] [ERROR] java.lang.IllegalStateException: Unable to process transformation: DOMAIN_REMOVE_SUBDOMAINS(full_hostname) for domain_without_subdomains because Unable to parse: DOMAIN_REMOVE_SUBDOMAINS(full_hostname) due to: EXACT with relevant variables full_hostname=www.aliexpress.com at org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:73) ~[stormjar.jar:?] at org.apache.metron.common.configuration.FieldTransformer.transform(FieldTransformer.java:139) ~[stormjar.jar:?] at org.apache.metron.common.configuration.FieldTransformer.transformAndUpdate(FieldTransformer.java:159) ~[stormjar.jar:?] at org.apache.metron.parsers.ParserRunnerImpl.applyFieldTransformations(ParserRunnerImpl.java:299) ~[stormjar.jar:?] at org.apache.metron.parsers.ParserRunnerImpl.processMessage(ParserRunnerImpl.java:254) ~[stormjar.jar:?] at org.apache.metron.parsers.ParserRunnerImpl.lambda$execute$0(ParserRunnerImpl.java:151) ~[stormjar.jar:?] at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0_112] at org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:150) ~[stormjar.jar:?] at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257) [stormjar.jar:?] at org.apache.storm.daemon.executor$fn__10252$tuple_action_fn__10254.invoke(executor.clj:735) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.daemon.executor$mk_task_receiver$fn__10171.invoke(executor.clj:466) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.disruptor$clojure_handler$reify__9685.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.daemon.executor$fn__10252$fn__10265$fn__10320.invoke(executor.clj:855) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.util$async_loop$fn__553.invoke(util.clj:484) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: org.apache.metron.stellar.dsl.ParseException: Unable to parse: DOMAIN_REMOVE_SUBDOMAINS(full_hostname) due to: EXACT with relevant variables full_hostname=www.aliexpress.com at org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:173) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.CachingStellarProcessor.parseUncached(CachingStellarProcessor.java:161) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.CachingStellarProcessor.parse(CachingStellarProcessor.java:155) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:50) ~[stormjar.jar:?] ... 18 more Caused by: java.lang.NoSuchFieldError: EXACT at org.apache.metron.guava.17.0.net.InternetDomainName.findPublicSuffix(InternetDomainName.java:173) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.guava.17.0.net.InternetDomainName.<init>(InternetDomainName.java:158) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.guava.17.0.net.InternetDomainName.from(InternetDomainName.java:213) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.dsl.functions.NetworkFunctions.toDomainName(NetworkFunctions.java:271) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.dsl.functions.NetworkFunctions.access$000(NetworkFunctions.java:34) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.dsl.functions.NetworkFunctions$RemoveSubdomains.apply(NetworkFunctions.java:86) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.dsl.BaseStellarFunction.apply(BaseStellarFunction.java:30) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.CachingStellarProcessor.parseUncached(CachingStellarProcessor.java:161) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.stellar.common.CachingStellarProcessor.parse(CachingStellarProcessor.java:155) ~[dep-stellar-common-0.7.2-uber-6958fda1-1cb6-4e03-80ba-c5171221d03a.jar.1566935872000:?] at org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:50) ~[stormjar.jar:?] ... 18 more ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
