[
https://issues.apache.org/jira/browse/METRON-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913686#comment-16913686
]
Michael Miklavcic commented on METRON-2230:
-------------------------------------------
The following config option needs to be added to global.json in order for the
[typsquatting|[https://github.com/apache/metron/tree/master/use-cases/typosquat_detection]]
use case to work.
{code:java}
"object.cache.max.file.size" : 5000000
{code}
Failure to provide the larger maximum file size will result in the following
error:
{code:java}
[Stellar]>>> BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'),
'gogle')
[!] Unable to parse:
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle') due to:
File at path '/tmp/reference/alexa10k_filter.ser' is larger than the configured
max file size of 1048576
org.apache.metron.stellar.dsl.ParseException: Unable to parse:
BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle') due to:
File at path '/tmp/reference/alexa10k_filter.ser' is larger than the configured
max file size of 1048576
at
org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:166)
at
org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154)
at
org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.executeStellar(DefaultStellarShellExecutor.java:407)
at
org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.execute(DefaultStellarShellExecutor.java:257)
at
org.apache.metron.stellar.common.shell.cli.StellarShell.execute(StellarShell.java:359)
at org.jboss.aesh.console.AeshProcess.run(AeshProcess.java:53)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: File at path
'/tmp/reference/alexa10k_filter.ser' is larger than the configured max file
size of 1048576
at
org.apache.metron.enrichment.cache.ObjectCache$Loader.load(ObjectCache.java:72)
at
org.apache.metron.enrichment.cache.ObjectCache$Loader.load(ObjectCache.java:46)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:3366)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2039)
at
java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2037)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2020)
at
com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
at
com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:67)
at
org.apache.metron.enrichment.cache.ObjectCache.get(ObjectCache.java:82)
at
org.apache.metron.enrichment.stellar.ObjectGet.apply(ObjectGet.java:66)
at
org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664)
at
org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259)
at
org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151)
... 7 more
{code}
The default max file size is 1MB while the bloom filter in the use case is over
4MB.
{code:java}
# hdfs dfs -ls /tmp/reference/alexa10k_filter.ser
-rw-r--r-- 1 root hdfs 4710254 2019-08-22 19:16
/tmp/reference/alexa10k_filter.ser
{code}
> Update typosquatting use case to reflect new OBJECT_GET max file size
> constraint property in global config
> ----------------------------------------------------------------------------------------------------------
>
> Key: METRON-2230
> URL: https://issues.apache.org/jira/browse/METRON-2230
> Project: Metron
> Issue Type: Bug
> Reporter: Michael Miklavcic
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
