Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-25 Thread Simon Elliston Ball
The original string serves purposes well beyond debugging. Many users will need to be able to prove provenance to the raw logs in order to prove or prosecute an attack from an internal threat, or provide evidence to law enforcement or an external threat. As such, the original string is important.

Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-25 Thread Michel Sumbul
Depending on the source of data, it might be interesting to bypass a step that the user concider useless. For example if you have a source of data that dont need profiling and you want to have it ingested like the other source to allow the SOC analyst to use it in there analysis. To have

Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-25 Thread James Sirota
There is a way to wire the system to bypass enrichment and profiling, but you would then bypass a lot of key features of the system. It would be unwise to do that. 25.06.2018, 15:13, "Michel Sumbul" : > Hi Casey, > > Thats make completely sense. > Short question, if there is no enrichment or

Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-25 Thread Michel Sumbul
Hi Casey, Thats make completely sense. Short question, if there is no enrichment or no profiling, does the message still pass through the enrichment/profiling topic? If yes, do you think its possible to imagine a way that for messages that doesn't need enrichment or profiling to skip the topic