I’m all for adding extra stores, especially once we have separated indexing
topologies.
Druid (and therefore a ui based on superset) seems an obvious logical store to
me. That said, the scheme management starts to feel like it needs some thought
once we have enough range of schema sensitive sto
Simon,
With the risk of sounding like an heretic:
Is there any particular reason Metron still considers ES as the
"default"[1] fast access data store?
Sometimes I wonder if we wouldn't be better off leveraging schema evolution
friendly formats with UIs like SuperSets?
Probably not as fast as ES
Correct, nested objects in lucene indexes lead to sub-documents, which leads to
a massive drop in ingest and query rates, this is why the JSONMap parser for
example deliberately flattens the Metorn JSON object. Before this decision was
made, very early versions of OpenSOC nested enrichments for
Hi Ali,
We have recently faced some data sources that generate data in a nested
format. For example, AWS Cloudtrail generates data in the following
JSON
format:
{
"Records": [
{
"eventVersion": *"2.0"*,
"userIdentity": {
"type": *"IAMUser"*,
"principalId": *"
So Metron enrichment and indexer are not nested aware? Is there any plan to
add that to Metron in future?
Cheers,
Ali
On Fri, Dec 22, 2017 at 12:46 AM, Otto Fowler
wrote:
> I believe right now you have to flatten.
> The jsonMap parser does this.
>
>
> On December 21, 2017 at 08:28:13, Ali Nazem
I believe right now you have to flatten.
The jsonMap parser does this.
On December 21, 2017 at 08:28:13, Ali Nazemian (alinazem...@gmail.com)
wrote:
Hi all,
We have recently faced some data sources that generate data in a nested
format. For example, AWS Cloudtrail generates data in the followi
Hi all,
We have recently faced some data sources that generate data in a nested
format. For example, AWS Cloudtrail generates data in the following JSON
format:
{
"Records": [
{
"eventVersion": *"2.0"*,
"userIdentity": {
"type": *"IAMUser"*,
"principalId":