Github user JonZeolla commented on a diff in the pull request:
https://github.com/apache/metron-bro-plugin-kafka/pull/2#discussion_r152075056
--- Diff: scripts/Bro/Kafka/logs-to-kafka.bro ---
@@ -14,32 +14,37 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-##! load this script to enable log output to kafka
+
+##! Load this script to enable log output to kafka
module Kafka;
export {
+ ## Specify which :bro:type:`Log::ID` to exclude from being sent to
kafka.
##
- ## which log streams should be sent to kafka?
- ## example:
- ## redef Kafka::logs_to_send = set(Conn::Log, HTTP::LOG,
DNS::LOG);
+ ## Example: redef Kafka::logs_to_exclude = set(SSH::LOG);
+ const logs_to_exclude: set[Log::ID] &redef;
+
+ ## Specify which :bro:type:`Log::ID` to send to kafka.
##
+ ## Example: redef Kafka::logs_to_send = set(Conn::Log, DNS::LOG);
const logs_to_send: set[Log::ID] &redef;
}
event bro_init() &priority=-5
{
for (stream_id in Log::active_streams)
{
- if (stream_id in Kafka::logs_to_send)
- {
- local filter: Log::Filter = [
- $name = fmt("kafka-%s", stream_id),
- $writer = Log::WRITER_KAFKAWRITER,
- $config = table(["stream_id"] = fmt("%s",
stream_id))
- ];
+ if ( stream_id in Kafka::logs_to_exclude ||
+ (|Kafka::logs_to_send| > 0 && stream_id !in
Kafka::logs_to_send) )
--- End diff --
Yeah, that's valid, I have removed the check and simplify.
Yeah, I would prefer a default 'send everything' policy when someone loads
the package, as long as it's otherwise configured. That said, it will require
a bit of Metron testing to make sure that it can handle that. We don't
currently handle some of the less interesting logs that are on by default, like
packet filter or loaded scripts.
---
