[
https://issues.apache.org/jira/browse/METRON-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852199#comment-15852199
]
ASF GitHub Bot commented on METRON-682:
---------------------------------------
Github user ottobackwards commented on the issue:
https://github.com/apache/incubator-metron/pull/432
[vagrant@node1 tmp]$ /usr/metron/0.3.0/bin/flatfile_loader.sh -i
http://s3.amazonaws.com/alexa-static/top-1m.csv.zip -t enrichment -c t -e
./extractor.json -p 5 -b 128
Exception in thread "main" org.codehaus.jackson.map.JsonMappingException:
Can not instantiate value of type [map type; class java.util.LinkedHashMap,
[simple type, class java.lang.String] -> [simple type, class java.lang.Object]]
from JSON String; no single-String constructor/factory method (through
reference chain:
org.apache.metron.dataloads.extractor.ExtractorHandler["config"])
at
org.codehaus.jackson.map.deser.std.StdValueInstantiator._createFromStringFallbacks(StdValueInstantiator.java:379)
at
org.codehaus.jackson.map.deser.std.StdValueInstantiator.createFromString(StdValueInstantiator.java:268)
at
org.codehaus.jackson.map.deser.std.MapDeserializer.deserialize(MapDeserializer.java:244)
at
org.codehaus.jackson.map.deser.std.MapDeserializer.deserialize(MapDeserializer.java:33)
at
org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
at
org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
at
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
at
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
at
org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732)
at
org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
at
org.apache.metron.dataloads.extractor.ExtractorHandler.load(ExtractorHandler.java:70)
at
org.apache.metron.dataloads.extractor.ExtractorHandler.load(ExtractorHandler.java:75)
at
org.apache.metron.dataloads.extractor.ExtractorHandler.load(ExtractorHandler.java:78)
at
org.apache.metron.dataloads.nonbulk.flatfile.SimpleEnrichmentFlatFileLoader.main(SimpleEnrichmentFlatFileLoader.java:49)
at
org.apache.metron.dataloads.nonbulk.flatfile.SimpleEnrichmentFlatFileLoader.main(SimpleEnrichmentFlatFileLoader.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
[vagrant@node1 tmp]$
with extractor.json of:
```
{
"config" :
"columns" : {
"domain" : 1,
"rank" : 0
}
,"indicator_column" : "domain"
,"type" : "alexa"
,"separator" : ","
},
"extractor" : "CSV"
}
```
> Unify and Improve the Flat File Loader
> --------------------------------------
>
> Key: METRON-682
> URL: https://issues.apache.org/jira/browse/METRON-682
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
>
> Currently the flat file loader is deficient in a couple ways:
> * It only supports importing local data despite there being a separate,
> poorly named, application which supports importing enrichment via MapReduce
> called threat_intel_loader.sh
> * It does not support local imports from HDFS
> * It does not support local imports from URLs
> * It does not support importing zipped archives locally
> * You cannot import more than one file at once
> This JIRA will:
> * Unify the MapReduce and local imports into one program and allow the user
> to specify the import mode with a CLI flag
> * Support local imports from HDFS and URLs
> * Support local imports from zipped files
> * Support importing more than one file at once
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
