Rendiyono Wahyu Saputro created METRON-2159:
-----------------------------------------------
Summary: BLOOM_EXISTS(OBJECT_GET()) Cannot work in SENSOR
ENRICHMENT CONFIG
Key: METRON-2159
URL: https://issues.apache.org/jira/browse/METRON-2159
Project: Metron
Issue Type: Bug
Affects Versions: 0.7.1
Environment: Operating System
- CentOS Linux release 7.6.1810 (Core)
Hortonworks Software Used
- Hortonworks Data Platform 2.6.5.0-292
- Hortonworks Data Flow 3.1.0.0
- Hortonworks CyberSecurity Platform 1.9.1.0
Software Used
- Apache Kafka 1.0.0
- Apache Storm 1.1.0
- Apache Nifi 1.5.0
- Apache Metron 0.7.1.1.9.1.0
- HDFS 2.7.3
- HBase 1.1.2
Hardware Used :
4 (four) node using VMWare Virtualization, each using 4 core, 15.51 GB RAM,
Storage 17.98 GB
Reporter: Rendiyono Wahyu Saputro
Hello,
Hi there,
I'm following this tutorial (
[https://metron.apache.org/current-book/use-cases/typosquat_detection/index.html]
). And in section Generate the Bloom Filter, i am very sure this command is
working and return value true in Stellar command
{code:java}
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle'){code}
But, it does not work dan return an error when i put it in a Sensor at Apache
Metron's Management -> RAW JSON -> SENSOR ENRICHMENT CONFIG. And i'm using
ElasticSearch so i checked ELASTICSEARCH ENABLED.
How i get the error, i use ElasticSearch HEAD as Google Chrome Addons (
[https://chrome.google.com/webstore/detail/elasticsearch-head/ffmkiejjmecolpfloofpjologoblkegm]
). And select error_index_ that have been created after Apache Metron send
index to ElasticSearch.
The Error :
{
* "_index": "error_index_2019.06.19.10",
* "_type": "error_doc",
* "_id": "AWtt2DrFshKy3KgUuMHX",
* "_version": 1,
* "_score": 1,
* "_source": {
** "exception": "java.lang.IllegalStateException: ENRICHMENT error with
stellar failed: Unable to parse:
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due
to: com.esotericsoftware.kryo.KryoException: Unable to find class:
Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter)
filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant
variables test_domain=gogle",
** "failed_sensor_type": "error",
** "stack": "java.lang.IllegalStateException: ENRICHMENT error with stellar
failed: Unable to parse:
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due
to: com.esotericsoftware.kryo.KryoException: Unable to find class:
Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter)
filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant
variables test_domain=gogle at
org.apache.metron.enrichment.parallel.ParallelEnricher.lambda$apply$0(ParallelEnricher.java:195)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) Caused by:
org.apache.metron.stellar.dsl.ParseException: Unable to parse:
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due
to: com.esotericsoftware.kryo.KryoException: Unable to find class:
Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter)
filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant
variables test_domain=gogle at
org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:173)
at
org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154)
at
org.apache.metron.enrichment.adapters.stellar.StellarAdapter.process(StellarAdapter.java:118)
at
org.apache.metron.enrichment.adapters.stellar.StellarAdapter.enrich(StellarAdapter.java:190)
at
org.apache.metron.enrichment.adapters.stellar.StellarAdapter.enrich(StellarAdapter.java:43)
at
org.apache.metron.enrichment.parallel.EnrichmentCallable.apply(EnrichmentCallable.java:64)
at
org.apache.metron.enrichment.parallel.EnrichmentCallable.apply(EnrichmentCallable.java:30)
at
com.github.benmanes.caffeine.cache.LocalCache.lambda$statsAware$0(LocalCache.java:143)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2039)
at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2037)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2020)
at
com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
at
com.github.benmanes.caffeine.cache.LocalManualCache.get(LocalManualCache.java:54)
at
org.apache.metron.enrichment.parallel.ParallelEnricher.lambda$apply$0(ParallelEnricher.java:182)
... 4 more Caused by:
org.apache.metron.guava.enrichment.util.concurrent.UncheckedExecutionException:
com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization
trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter
(org.apache.metron.stellar.common.utils.BloomFilter) at
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.get(LocalCache.java:2256)
at
org.apache.metron.guava.enrichment.cache.LocalCache.get(LocalCache.java:3980)
at
org.apache.metron.guava.enrichment.cache.LocalCache.getOrLoad(LocalCache.java:3984)
at
org.apache.metron.guava.enrichment.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4868)
at org.apache.metron.enrichment.stellar.ObjectGet.apply(ObjectGet.java:110) at
org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664)
at
org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259)
at
org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151)
... 17 more Caused by: com.esotericsoftware.kryo.KryoException: Unable to find
class: Serialization trace: bits
(org.apache.metron.guava.enrichment.hash.BloomFilter) filter
(org.apache.metron.stellar.common.utils.BloomFilter) at
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156)
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118) at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:759) at
com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132) at
com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790) at
org.apache.metron.common.utils.SerDeUtils.fromBytes(SerDeUtils.java:262) at
org.apache.metron.enrichment.stellar.ObjectGet$Loader.load(ObjectGet.java:87)
at
org.apache.metron.enrichment.stellar.ObjectGet$Loader.load(ObjectGet.java:72)
at
org.apache.metron.guava.enrichment.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3579)
at
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.loadSync(LocalCache.java:2372)
at
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2335)
at
org.apache.metron.guava.enrichment.cache.LocalCache$Segment.get(LocalCache.java:2250)
... 24 more Caused by: java.lang.ClassNotFoundException: at
java.net.URLClassLoader.findClass(URLClassLoader.java:381) at
java.lang.ClassLoader.loadClass(ClassLoader.java:424) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at
java.lang.ClassLoader.loadClass(ClassLoader.java:357) at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:348) at
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
... 39 more ",
** "hostname": "HCP1-Master1",
** "source:type": "error",
** "raw_message":
"\{"":{"test_domain":null,"url":"www.gogle.com"},"source.type":"typo"}",
** "error_hash":
"8219f37f7fd3f84b8dc6a2e04dd68b6139795b8e8fe4cc072ae9344ea559d1dc",
** "error_type": "enrichments_error",
** "guid": "6d295227-aa41-4a99-95e8-46dc5a938e1f",
** "message": "ENRICHMENT error with stellar failed: Unable to parse:
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due
to: com.esotericsoftware.kryo.KryoException: Unable to find class:
Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter)
filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant
variables test_domain=gogle",
** "timestamp": 1560916000031}
}
Can you help me how to figure out what is going on. Thank in advance
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
