Re: [Zeek-Dev] docs.zeek.org

2019-01-17 Thread Jon Siwek
On Fri, Jan 11, 2019 at 10:20 AM Jon Siwek wrote: > (note that the "release" version of the manual still lives on > zeek.org as there's not yet any release that can be build on RTD). Both release and master versions are on RTD (docs.zeek.org) now and build from a new zeek-doc

Re: [Zeek-Dev] adding credits to the schema for package metadata

2019-01-04 Thread Jon Siwek
On Thu, Jan 3, 2019 at 5:04 PM Vern Paxson wrote: > [package] > credit = Originally written by A. Sacker . JSON > support added by W00ter (Acme Corporation). So first I have to question the use-case: who or what will need this information and how often? This is not

[Zeek-Dev] docs.zeek.org

2019-01-10 Thread Jon Siwek
FYI, https://docs.zeek.org is now live with the Zeek manual hosted by a Read the Docs custom domain (originally was hosted at www.zeek.org/sphinx-git). The broker and bro-pkg docs are also now available as sub-projects there: https://docs.zeek.org/projects/broker

Re: [Zeek-Dev] docs.zeek.org

2019-01-11 Thread Jon Siwek
On Fri, Jan 11, 2019 at 9:04 AM Robin Sommer wrote: > > Can we keep old links into the documentation working through > redirects? Yes, those should already be set up, but let me know if I missed anything (note that the "release" version of the manual still lives on zeek.org as there's not yet

Re: [Zeek-Dev] adding credits to the schema for package metadata

2019-01-11 Thread Jon Siwek
On Fri, Jan 4, 2019 at 4:05 PM Jon Siwek wrote: > I'm still cool with documenting an optional "credits" field for the > package metadata I went ahead and documented it [1] as I was updating other things there. Feel free to PR any language tweaks. - Jon [1] https://docs.z

Re: [Zeek-Dev] docs.zeek.org

2019-01-11 Thread Jon Siwek
On Fri, Jan 11, 2019 at 5:03 PM Robin Sommer wrote: > Would it be worth aiming to do that update with the next 2.6.x patch > release? Would be nice to get the modern look for the release version, > too. Not sure how worth it -- feel a bit odd/unmotivated preparing a pure-cosmetic feature for a

Re: [Zeek-Dev] adding credits to the schema for package metadata

2019-01-04 Thread Jon Siwek
On Fri, Jan 4, 2019 at 1:48 PM Vern Paxson wrote: > > It's maybe something packages.zeek.org would show, but I don't see how > > that would be better than looking at the "contributors" stats already > > compiled by GitHub from author info encoded directly into git commits. > > The problem with

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-20 Thread Jon Siwek
On Tue, Feb 19, 2019 at 10:55 PM Robin Sommer wrote: > > On Tue, Feb 19, 2019 at 08:28 -0600, Jonathan Siwek wrote: > > > we pick points in time as an "event" and associate various bits of > > data with them, then users declare their interest in some subset of > > that data. Most natural way for

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-20 Thread Jon Siwek
On Tue, Feb 19, 2019 at 1:27 PM Vern Paxson wrote: > > Which particular user errors were a concern? > > The main one being when there's an API change that's *not* backward > compatible, and the user doesn't update their scripts to be in conformance > with it as is required. Clearly something

Re: [Zeek-Dev] Using a BiF across C++ and Zeek Policy

2019-03-13 Thread Jon Siwek
On Wed, Mar 13, 2019 at 11:48 AM Jan Grashöfer wrote: > > On 12/03/2019 17:15, zeo...@gmail.com wrote: > > I am working on improving the btests for the kafka writer plugin with the > > goal of validating some logic in KafkaWriter::DoInit. The best approach > > that I have so far is to write a BiF

Re: [Zeek-Dev] move krb5_init_context out of KRB.cc

2019-03-11 Thread Jon Siwek
On Mon, Mar 11, 2019 at 12:29 AM Song wrote: > I'm trying to write a Kerberos GSSAPI decryption support analyzer. Currently > krb5_init_context() is called to get a krb5_context inside KRB > analyzer. I think it's a good thing to share the context among all the > components that need to call

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-06 Thread Jon Siwek
On Wed, Feb 6, 2019 at 10:51 AM Seth Hall wrote: > One thing I've been thinking about a little bit is how much we're > concerning ourselves with maintaining perfect backward compatibility and > if there is some benefit to breaking a bit of backward compatibility for > something truly nicer?

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-06 Thread Jon Siwek
On Wed, Feb 6, 2019 at 1:30 PM Vlad Grigorescu wrote: > I _think_ I like Seth's idea of records, but I'm still thinking it through. > It would formalize a growing trend towards moving more parameters into > records anyway. If we're worried about backwards compatibility, then maybe we > have a

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-06 Thread Jon Siwek
On Tue, Feb 5, 2019 at 7:40 PM Robin Sommer wrote: > The following would be even worst in terms of confusion: > > global my_event: event(a: string, b: string); > event my_event(b: string) > > Now I need to know if the language goes by order of parameters or by > parameter name. The

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-19 Thread Jon Siwek
On Mon, Feb 18, 2019 at 6:46 PM Vern Paxson wrote: > Suppose "event foo(a: string)" is an event to which we want to add > a second parameter, b: string. We could express this in event.bif as: > > event foo%(a: string%) > event foo%(a: string, b: string%); Nice idea. I also

[Zeek-Dev] support for event handlers using a subset of parameters

2019-01-31 Thread Jon Siwek
Just wanted to draw more attention/feedback to a proof-of-concept patch I just made: https://github.com/zeek/zeek/pull/262 Basically lets us do this: global my_event: event(a: count, b: string); event my_event(b: string) { print "my_event", b; } Which is good because: * user

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-01 Thread Jon Siwek
On Thu, Jan 31, 2019 at 6:29 PM Vern Paxson wrote: > > * user doesn't care about parameter 'a', so they shouldn't have to list it > > * it makes it easier for to deprecate/change event parameters > > This seems like a pretty niche pair of benefits. Is there a compelling > use-case that's

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-01 Thread Jon Siwek
On Fri, Feb 1, 2019 at 10:24 AM Robin Sommer wrote: > On Thu, Jan 31, 2019 at 16:29 -0800, Vern Paxson wrote: > > > > global my_event: event(a: count, b: string); > > > event my_event(b: string) > > > { print "my_event", b; } > > it *looks* like an error to not have matching >

Re: [Zeek-Dev] support for event handlers using a subset of parameters

2019-02-01 Thread Jon Siwek
On Fri, Feb 1, 2019 at 12:59 PM Vern Paxson wrote: > I don't see how it helps with > deprecating existing parameters (which seems would be better served with > some sort of attribute), Support for in parameters is part of the changes. But if we don't allow the user to immediately remove the

Re: [Zeek-Dev] CFLAGS/CXXFLAGS bug with configure

2019-05-29 Thread Jon Siwek
On Wed, May 29, 2019 at 10:30 AM Jim Mellander wrote: > There appear to be several alternatives to allow supplied flags to override > the defaults, the most reasonable appearing to be modifying the build > commands to apply CFLAGS/CXXFLAGS after the defaults. > > I'm not a cmake maven, but it

Re: [Zeek-Dev] Dealing with directional out-of-order packet?

2019-06-25 Thread Jon Siwek
On Mon, Jun 24, 2019 at 6:34 PM Song wrote: > So my question is: is this other-side gap information available in > Binpac? Is it already available in the form you'd prefer? Possibly not. You can maybe look at whether TCP_Analyzer::IsPartial() and TCP_Analyzer::HadGap() could be used to detect

Re: [Zeek-Dev] Zeek and the myricom package plugin

2019-07-12 Thread Jon Siwek
On Fri, Jul 12, 2019 at 3:46 PM Michael Dopheide wrote: > Background: We like to run 'master', but with the name change it broke > too many things and we had to stick to 2.6.2 for the time being. Since > then, I've started trying to convert our ansible scripts and prepare to > make the jump.

Re: [Zeek-Dev] Sending Packets via Broker

2019-08-26 Thread Jon Siwek
On Mon, Aug 26, 2019 at 6:29 AM Vlad Grigorescu wrote: > > Master has code for setting up the cluster framework with time machine nodes, > and is_external_connection is a BIF that determines if a connection has been > received from an external source, but in Broker, I don't see how I would send

[Zeek-Dev] Zeek 3.0.0+ "master" versioning process change

2019-07-24 Thread Jon Siwek
I'm suggesting a small change to our usual development and versioning process of the "master" branch: replace post-release versioning with pre-release versioning. E.g. once we enter the 3.0.0 beta period, "master" will start using versions like 3.1.0-alpha.X where the old process would have used

Re: [Zeek-Dev] Zeek 3.0.0+ "master" versioning process change

2019-07-24 Thread Jon Siwek
On Wed, Jul 24, 2019 at 6:02 PM Johanna Amann wrote: > > Actually, thinking about it some more - could we just not have the > -alpha (or -dev) label, and go back to how it was before - with a > changed meaning? > > so - just 3.1.0-[commit-number] for the development builds. Our versioning script

[Zeek-Dev] maybe migrating Zeek CI from Travis to Cirrus

2019-12-09 Thread Jon Siwek
I experimented last week moving the CI config to Cirrus and found several benefits to making that switch. Example build in my Zeek fork: https://cirrus-ci.com/build/6738081318699008 Notable differences from the Travis config: * Free for open-source projects, can allocate up to 8 cpus (we get 2

Re: [Zeek-Dev] alternative to open_log_file ?

2019-10-17 Thread Jon Siwek
On Thu, Oct 17, 2019 at 4:08 PM Aashish Sharma wrote: > > I just noticed (rather starting to notice :) open_log_file was depricated in > 2.6 and is removed in 3.0.0 > > What would be suggested alternative for this ? Either the logging framework or just use open() directly: open_log_file() didn't

[Zeek-Dev] Zeek Supervisor: designing client and log archival behavior

2020-06-30 Thread Jon Siwek
Looking for feedback on the design/plan for these two Zeek Supervisor components: * https://github.com/zeek/zeek/wiki/Zeek-Supervisor-Client * https://github.com/zeek/zeek/wiki/Zeek-Supervisor-Log-Handling - Jon ___ Zeek-Dev mailing list

Re: [Zeek-Dev] Log archival (Re: Zeek Supervisor: designing client and log archival) behavior

2020-07-01 Thread Jon Siwek
On Wed, Jul 1, 2020 at 1:59 AM Robin Sommer wrote: > > > Log::default_rotation_dir > > Seems we should then set this to "." by default, and have the cluster > framework override it. Yes, exactly. > Once moved, I suppose we would continue to optionally run a > post-processor, right? For a

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-13 Thread Jon Siwek
On Fri, Jul 10, 2020 at 11:00 AM Bob Murphy wrote: > The development I was doing sometimes required me to examine the debug > messages from different threads in the chronological order they were > generated. But if I understand it correctly, the threading framework’s > logging doesn’t

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Jon Siwek
On Tue, Jul 14, 2020 at 11:56 AM Bob Murphy wrote: > The code you show should give correct ordering on when Foo() and Bar() finish. Wondering what's meant by "correct ordering" here. Bar() can finish before Foo() and yet debug.log can report "I did Foo" before "I did Bar" for whatever

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Jon Siwek
On Tue, Jul 14, 2020 at 8:05 AM Bob Murphy wrote: > My current implementation does just use a mutex to control access to the > output file, and reports the time of generation. I was also trying to break down a couple distinct requirements and wondered if that actually covers the 2nd: (1) Fix

Re: [Zeek-Dev] Zeek Supervisor Command-Line Client

2020-06-30 Thread Jon Siwek
On Tue, Jun 30, 2020 at 6:35 AM Seth Hall wrote: > I'm really starting to think that the business logic for > correctly starting and stopping a cluster should be fully implemented in > the supervisor script. The zeekc tool could then just be a dumb tool > that says to start and stop and doesn't

Re: [Zeek-Dev] Zeek Supervisor Command-Line Client

2020-06-19 Thread Jon Siwek
On Fri, Jun 19, 2020 at 1:38 AM Robin Sommer wrote: > think we also want their state controllable from the client as well, > so that one can have an orderly shutdown of a multi-system cluster > without loss of data (e.g., one probably wants to shutdown workers > first to collect remaining log

Re: [Zeek-Dev] Zeek Supervisor Command-Line Client

2020-06-18 Thread Jon Siwek
On Thu, Jun 18, 2020 at 7:45 AM Vlad Grigorescu wrote: > My main concern was Broker version incompatibilities between the > newly-installed zcl, and the running cluster, which I think would be > addressed by that (i.e. to stop a cluster, you stop the supervisor service on > the manager, and

Re: [Zeek-Dev] Zeek Supervisor Command-Line Client

2020-06-18 Thread Jon Siwek
On Thu, Jun 18, 2020 at 12:11 AM Robin Sommer wrote: > For (1), the above applies: we'll rely on standard sysadmin processes > for updating. That means you'd use "zeekcl" to shutdown the cluster > processes, then run "yum update" (or whatever), then use "zeekcl" > again to start things up again.

[Zeek-Dev] Zeek Supervisor Command-Line Client

2020-06-17 Thread Jon Siwek
Don't recall any basic "project infrastructure" discussions happening yet for the upcoming replacement/alternative for ZeekControl that we want to introduce in Zeek 3.2 (roadmap/design links found at [1]), so here's starting questions. # What to Name It ? Suggestion: `zeekcl`, Zeek

Re: [Zeek-Dev] Use zeek scripts only with the "manager" in a cluster configuration

2020-06-10 Thread Jon Siwek
On Wed, Jun 10, 2020 at 8:14 AM Davide Robusto wrote: > Is it possible to inhibit the use of code fragments for workers? > Zeek scripts can only be used by Zeek's "manager" or "logger", > so as to prevent "workers" from doing unwanted actions? If yes, is also > possible to make "worker" > able