from:"Robin Sommer"


[ 
https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=15300#comment-15300
 ] 

Robin Sommer commented on BIT-867:
--

{noformat}
// Not considering routing presence bit since it's deprecated...
{noformat}

Would it hurt to add that? Looks like it's just another length adjustment if 
present?



 GRE support
 ---

 Key: BIT-867
 URL: https://bro-tracker.atlassian.net/browse/BIT-867
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Robin Sommer
 Fix For: 2.3


 Should be rather easy to add support for GRE tunnels now.



--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1115) topic/jazoff/suppression


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1115:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/jazoff/suppression
 

 Key: BIT-1115
 URL: https://bro-tracker.atlassian.net/browse/BIT-1115
 Project: Bro Issue Tracker
  Issue Type: Patch
  Components: Bro
Affects Versions: 2.2
Reporter: Justin Azoff





--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1116) topic/jsiwek/libmagic-integration


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1116:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/jsiwek/libmagic-integration
 -

 Key: BIT-1116
 URL: https://bro-tracker.atlassian.net/browse/BIT-1116
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: Bro
Reporter: Jon Siwek
 Fix For: 2.3


 This branch is in bro, 3rdparty, bromagic, bro-testing, and 
 bro-testing-private repos.  It integrates libmagic 5.16 into Bro as a CMake 
 ExternalProject, which requires CMake = 2.8.0, so that one does not have to 
 install libmagic to build bro.
 Resolves BIT-, BIT-1096.



--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-867) GRE support


 [ 
https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-867:
-

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 GRE support
 ---

 Key: BIT-867
 URL: https://bro-tracker.atlassian.net/browse/BIT-867
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Robin Sommer
 Fix For: 2.3


 Should be rather easy to add support for GRE tunnels now.



--
This message was sent by Atlassian JIRA
(v6.2-OD-07-028#6211)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1108) Add broctl option to set PF_RING cluster type

2014-01-13 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1108:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Add broctl option to set PF_RING cluster type
 -

 Key: BIT-1108
 URL: https://bro-tracker.atlassian.net/browse/BIT-1108
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
Reporter: Daniel Thayer
 Fix For: 2.3


 Currently, when using PF_RING, broctl chooses the PF_RING
 cluster type by setting the environment variable
 PCAP_PF_RING_USE_CLUSTER_PER_FLOW.  In order to use a
 different cluster type, we would need to set a different
 environment variable (the PF_RING-aware libpcap does not
 look at the actual value of the environment variable,
 just whether the variable is defined or not), but there is
 no option in broctl to do this.
 To address this issue, a new broctl option PFRINGClusterType
 can be added, then a user could change the value of this 
 option to choose a different PF_RING cluster type (and the 
 broctl pf_ring plugin would set the appropriate env. variable).  
 The allowed values of this new broctl option would be:
 2-tuple, 4-tuple, 5-tuple, tcp-5-tuple, round-robin, 
 or 6-tuple (this one corresponds to the current
 cluster type used by broctl).  By default, PFRINGClusterType 
 would be set to 6-tuple.



--
This message was sent by Atlassian JIRA
(v6.2-OD-07-027#6211)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1109) topic/dnthayer/doc-updates

2013-12-24 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1109:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/dnthayer/doc-updates
 --

 Key: BIT-1109
 URL: https://bro-tracker.atlassian.net/browse/BIT-1109
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro, BroControl
Reporter: Daniel Thayer
 Fix For: 2.3


 This branch (in bro and broctl repos) includes miscellaneous documentation
 fixes.



--
This message was sent by Atlassian JIRA
(v6.2-OD-05-4#6207)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))

2013-12-20 Thread Robin Sommer



On Thu, Dec 19, 2013 at 18:55 -0500, you wrote:

 What's the reason for supporting both static and dynamic plugin types?  

That's exactly what I haven't really made up my mind about yet. :) I
think there's benefit to having a single Bro binary that comes with
all the standard functionality. One piece is portability: dynamic
linking may not be feasible/possible on some platforms (like tiny
devices, or exotic OSs where our cmake setup may fail to do the right
thing). And I generally like the notion of having just a single binary
with all the standard code included; means less can go wrong (like
version mismatches, etc.)

In terms of performance, I wouldn't be too worried actually, although
it's something that needs testing.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))

2013-12-19 Thread Robin Sommer



On Wed, Dec 18, 2013 at 12:20 -0500, you wrote:

 I just build bro, cd into the build directory, source in the bro-path-dev.sh 
 script and run Bro.

Ah, I see. It's something else than I thought: a left-over from the
earlier version that isn't needed anymore. Removed.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))

2013-12-19 Thread Robin Sommer



On Wed, Dec 18, 2013 at 21:07 -0500, you wrote:

 /tmp/bro/src/util.h:24:10: fatal error: 'magic.h' file not found

I didn't consider Bro's CXX_FLAGS. I think I've fixed that, please try
again.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))

2013-12-19 Thread Robin Sommer



On Thu, Dec 19, 2013 at 10:23 -0500, you wrote:

 * Would a section on testing be appropriate?  Both btest and unit
 testing might be useful for plugins.

Ack, that's a good point, the init-plugin script could put a basic
setup in place for that, maybe even with a first test making sure
things compile.

 * A short section explaining how / when to modify CMakeLists.txt might
 be useful.

Yeah, likewise agreed. Indeed tthe documentation needs quite a bit
more material to get people actually started without having to browse
a ton of other code first. I'll leave that for later though once we've
fleshed this all fully out.

 * Should plugins be allowed to link to additional libraries?

Yes, definitly. My thinking is that the plugin author will extend the
CMakeIndex.txt with the corresponding pieces, including compile-time
logic to figure out if it's available. However, if the binary module
aims to link against a lib that's not available at runtime where Bro
executes, then I don't think there's much more we can do than fail
loading the plugin: the dlopen will fail (iirc, Bro currently aborts
in that case, I'm not sure if it should proceed without?)

Thanks for the feedback. From chatting with Seth the other day, I took
two more suggestions away:

- I'm coming around that the BRO_PLUGIN_* macros aren't the best way
  of doing things. My main motivation for using them was hiding
  implementation details of the plugin API so that we can more easily
  change things without breaking existing code. However, it seems they
  are putting too much constraints on the plugin writer and/or, if one
  needs to get around them, require a lot of digging into the
  internals. So I'm mulling over creating a (simpler) C++ API to the
  Plugin class that can be used directly.

- The static and dynamic plugins could be unified further. It's 
  unclear what the right default is for shipping plugins that provide
  standard functionality, but it would be nice in any case if we could
  just flip a switch to change between static and dynamic builds for
  the in-tree stuff.

Robin


-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))

2013-12-18 Thread Robin Sommer



On Wed, Dec 18, 2013 at 08:10 -0500, you wrote:

   error in ./plugins, line 1: read failed with Is a directory

Doh. :) Not sure how to reproducee though. How exactly are you running
it? Are you setting BRO_PLUGIN_PATH, and if so, how?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] Dynamic plugin model (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-plugins-2.3: Start of a plugin writing how-to. (87a1618))

2013-12-16 Thread Robin Sommer

 the
 +``@load-plugin qualified-plugin-name`` directive (e.g.,
 +``@load-plugin Demo::Rot13``).
 +
 +``bro -N`` shows activated and found yet unactivated plugins
 +separately. Note that plugins compiled statically into Bro are always
 +activated, and hence show up as such even in bare mode.
 +
 +.. todo::
 +
 +Is this the right activation model?
 +
 +
 +Plugin Component
 +
 +
 +The following gives additional information about providing individual
 +types of functionality via plugins. Note that a single plugin can
 +provide more than one type. For example, a plugin could provide
 +multiple protocol analyzers at once; or both a logging backend and
 +input reader at the same time.
 +
 +We now walk briefly through the specifics of providing a specific type
 +of functionality (a *component*) through plugin. We'll focus on their
 +interfaces to the plugin system, rather than specifics on writing the
 +corresponding logic (usually the best way to get going on that is to
 +start with an existing plugin providing a corresponding component and
 +adapt that). We'll also point out how the CMake infrastructure put in
 +place by the ``init-plugin`` helper script ties the various pieces
 +together.
 +
 +Bro Scripts
 +---
 +
 +Scripts are easy: just put them into ``scripts/``, as described above.
 +The CMake infrastructure will automatically install them, as well
 +include them into the source and binary plugin distributions.
 +
 +Builtin Language Elements
 +-
 +
 +Functions
 +TODO
 +
 +Events
 +TODO
 +
 +Types
 +TODO
 +
 +Protocol Analyzers
 +--
 +
 +TODO.
 +
 +File Analyzers
 +--
 +
 +TODO.
 +
 +Logging Writer
 +--
 +
 +Not yet implemented.
 +
 +Input Reader
 +
 +
 +Not yet implemented.
 +
 +Packet Sources
 +--
 +
 +Not yet implemented.
 +
 +Packet Dumpers
 +--
 +
 +Not yet implemented.
 +
 +Documenting Plugins
 +===
 +
 +..todo::
 +
 +Integrate all this with Broxygen.
 +
 +
 +
 
 ___
 bro-commits mailing list
 bro-comm...@bro.org
 http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
 



-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Proposed IOSource reorg

2013-12-11 Thread Robin Sommer

As I'm working on the reorg, I propose to do the following:

- Remove flow sources completely for now. Per below, we should
  eventually turn them into a file analyzer and at it doesn't look
  worth the effort (nor the ugliness) to migrate them over to the
  new structure first only to throw them out later. I'd be
  surprised if anybody is using them anyways.

- Remove the secondary path from the packet-layer code. We have
  discussed this before and at that time decided for keeping the
  code; see https://bro-tracker.atlassian.net/browse/BIT-434

  However, I propose to go ahead and remove now because (1) it
  doesn't really fit the new structure of making the API (mostly)
  pcap-independent (it never really fit in well in the first
  place, and has made the code a lot more complex); (2)
  large-conns.bro seems to be the only actual use case, which we
  don't ship with 2.x anymore, and I'm not convinced that by
  itself warrants a separate data path (can we find a different
  solution to the problem?); and (3) it would be quite a bit of
  additional effort to port the code and make sure it still works
  (we don't have any tests, not surprisingly).

Thoughts?

Robin

On Wed, Dec 04, 2013 at 11:12 -0500, you wrote:

 
 On Dec 3, 2013, at 1:07 PM, Robin Sommer ro...@icir.org wrote:
 
 src/iosource/sources/flow-src/*
 
 To document our conversation from yesterday, flow-src should probably
 be thrown out and the netflow analyzer turned into a file analyzer. 
 Extending the input framework to be able to open raw sockets would
 then enable us to create an input stream holding open a datagram
 socket and attach the netflow file analyzer to it.  This would
 simplify the whole thing and make it possible to reuse the netflow
 analyzer code because we could yank netflow directly off the wire with
 it too (pending some analyzer infrastructure re-architecting).
 
   .Seth 
 
 --
 Seth Hall
 International Computer Science Institute
 (Bro) because everyone has a network
 http://www.bro.org/
 





-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1105) /topic/jsiwek/misc-fixes


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1105:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 /topic/jsiwek/misc-fixes
 

 Key: BIT-1105
 URL: https://bro-tracker.atlassian.net/browse/BIT-1105
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro, Broccoli, BroControl
Affects Versions: git/master
Reporter: Jon Siwek
Priority: High
 Fix For: 2.3


 This is in bro, broccoli, and broctl.  It fixes various build/test/coverity 
 failures.
 The ref counting fix may be a pre-existing issue relevant to 2.2, but just 
 coincidentally exposed on one jenkins node now.



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1103) Memory leak in Bro Intel framework


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1103:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Memory leak in Bro Intel framework
 --

 Key: BIT-1103
 URL: https://bro-tracker.atlassian.net/browse/BIT-1103
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: 2.2
 Environment: Red Hat Enterprise Linux Server release 6.5
Reporter: Andrew Hoying
Assignee: Bernhard Amann
Priority: High
  Labels: intel, leak

 The policy/frameworks/intel/seen bro scripts have a memory leak. On my 
 moderately busy Bro installation I am leaking about a gig of memory a day per 
 worker process with the Intel framework enabled. I can replicate by adding 
 the following to the local.bro default script and then running through a 
 small PCAP with primarily dns, dhcp and syslog traffic.
 {{
 @load policy/frameworks/intel/seen
 redef Intel::read_files += {
 /usr/local/bro/spool/domain_suspicious.txt,
 };
 }}
 The intel file is in the following format, here's a few sample lines. It is 
 generated automatically by CIF:
 {{
 #fields indicator   indicator_type  meta.source meta.desc   
 meta.urlmeta.cif_impact meta.cif_severity   meta.cif_confidence
 mete-tools.biz  Intel::DOMAIN   CIF - need-to-know  spammed domain  
 http://www.spamhaus.org/query/dbl?domain=mete-tools.biz (public)- 
   -   95
 rttvxygkmwlqmq.net  Intel::DOMAIN   CIF - need-to-know  spammed 
 domain  http://www.spamhaus.org/query/dbl?domain=rttvxygkmwlqmq.net (public)  
   -   -   95
 podserveruho.comIntel::DOMAIN   CIF - need-to-know  spammed 
 domain  http://www.spamhaus.org/query/dbl?domain=podserveruho.com (public)
   -   -   95
 wwfcogdgntlxw.biz   Intel::DOMAIN   CIF - need-to-know  spammed 
 domain  http://www.spamhaus.org/query/dbl?domain=wwfcogdgntlxw.biz (public)   
   -   -   95
 }}
 I compiled bro with gperftool debug support and followed the instructions 
 here: http://www.bro.org/development/howtos/leaks.html. (Note, the 
 instructions are wrong on the flags for ./configure, you need to add 
 --enable-perftools-debug to get the -m option for bro)
 Here's the output from pprof top after running a PCAP trace with 10,000 
 packets. Running traces with more packets show a greater number of lost 
 objects in the same code locations.
 {{
 # pprof bin/bro /tmp/bro.24541.net_run-end.heap --inuse_objects --lines 
 --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10
 Using local file bin/bro.
 Using local file /tmp/bro.24541.net_run-end.heap.
 Welcome to pprof!  For help, type 'help'.
 (pprof) top
 Total: 4295 objects
 2150  50.1%  50.1% 2150  50.1% AsciiFormatter::ParseValue 
 /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:186
 2141  49.8%  99.9% 2141  49.8% copy_string 
 /usr/src/bro-2.2/src/util.cc:155
2   0.0% 100.0%2   0.0% re_alloc 
 /usr/src/bro-2.2/build/src/re-scan.cc:2287
1   0.0% 100.0%1   0.0% RE_parse 
 /usr/src/bro-2.2/build/src/re-parse.y:110
1   0.0% 100.0%1   0.0% RE_parse 
 /usr/src/bro-2.2/build/src/re-parse.y:133
0   0.0% 100.0% 2141  49.8% AsciiFormatter::ParseValue 
 /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:195
0   0.0% 100.0%4   0.1% Connection::NextPacket 
 /usr/src/bro-2.2/src/Conn.cc:259
0   0.0% 100.0%4   0.1% NetSessions::DispatchPacket 
 /usr/src/bro-2.2/src/Sessions.cc:189
0   0.0% 100.0%4   0.1% NetSessions::DoNextPacket 
 /usr/src/bro-2.2/src/Sessions.cc:709
0   0.0% 100.0%4   0.1% NetSessions::NextPacket 
 /usr/src/bro-2.2/src/Sessions.cc:247
 }}



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1106) Merge topic/bernhard/input-error-fixes

[
https://bro-tracker.atlassian.net/browse/BIT-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robin Sommer updated BIT-1106:
--

Resolution: Merged (was: Fixed)
Status: Closed (was: Merge Request)

Merge topic/bernhard/input-error-fixes
--

Key: BIT-1106
URL: https://bro-tracker.atlassian.net/browse/BIT-1106
Project: Bro Issue Tracker
Issue Type: Improvement
Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann

The branch topic/bernhard/input-error-fixes fixes a number of issues of the
input framework that all have to do with errors:
-First:
Due to architectural constraints, it is very hard for the input framework to
handle optional records. For an optional record, either the whole record has
to be missing, or all non-optional elements of the record have to be defined.
This information is not available to input readers after the records have
been unrolled into the threading types.
Behavior so far was to treat optional records like they are non-optional,
without warning. The patch changes this behavior to emit an error on
stream-creation (during type-checking) and refusing to open the file. I think
this is a better idea - the behavior so far was undocumented and unintuitive.
- Second:
For table and event streams, reader backend creation was done very early,
before actually checking if all arguments are valid. Initialization is moved
after the checks now - this makes a number of delete statements unnecessary.
Also - I suspect threads of failed input reader instances were not deleted
until shutdown
- Third:
Add a couple more consistency checks, e.g. checking if the destination value
of a table has the same type as we need. We did not check everything in all
instances, instead we just assigned the things without caring (which works,
but is not really desirable).
This change also exposed a few bugs in other testcases where table
definitions were wrong (did not respect $want_record)
- Fourth:
Improve error messages and write testcases for all error messages (I think).

--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1104) Add tracking for MSIE 11


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1104:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Add tracking for MSIE 11
 

 Key: BIT-1104
 URL: https://bro-tracker.atlassian.net/browse/BIT-1104
 Project: Bro Issue Tracker
  Issue Type: Patch
  Components: Bro
Affects Versions: 2.1
 Environment: Ubuntu
Reporter: Michael Stone
Assignee: Seth Hall
  Labels: analyzer

 MSIE 11.0 currently shows up as unknown browser.  It looks like MS might 
 have changed it's user agent string and doesn't include MSIE.  I added the 
 following to /usr/local/bro/share/bro/base/frameworks/software/main.bro
 just below the MSIE block and above the Safari block.
 else if ( /Trident\/7.0/ in uparsed_version ) 
{
if ( /rv:11\.0/ in unparsed_version ) {
software_name = MSIE;
v = [$major=11,$minor=0];
   }
}
 Disclaimer: I'm fairly new to working with Bro so this might not be the best 
 way, but it seems to be working for me.
 Thanks!



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1107) Documentation of BIFs that take variable number of arguments

2013-12-06 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14918#comment-14918
 ] 

Robin Sommer commented on BIT-1107:
---

The work-around of turning va_args function arguments into {{(...)}}}, along 
with a  manual textual description of how the parameters are supposed to look 
like in each case, would sound good to me.

Btw, I believe this is how Bro recognizes va_args functions: 

{noformat}
int check_and_promote_exprs(ListExpr* elements, TypeList* types)
{
[...]
if ( tl-length() == 1  (*tl)[0]-Tag() == TYPE_ANY )
return 1;
[...]
}
{noformat}

Would be nicer to have  some more explicit way some time.

 Documentation of BIFs that take variable number of arguments
 

 Key: BIT-1107
 URL: https://bro-tracker.atlassian.net/browse/BIT-1107
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Reporter: Daniel Thayer

 The function prototype for BIFs that take a variable number of 
 arguments appears in an altered form in the online documentation.
 Here is a comparison of how these functions appear in the source code, 
 versus what they look like in the online documentation:
 md5_hash%(...%)  ===  Type : function (va_args: any)
 order%(v: any, ...%)  ===  Type : function (va_args: any)
 sort%(v: any, ...%)  ===  Type : function (va_args: any)
 cat_sep%(sep: string, def: string, ...%)  ===  Type :  function (va_args: 
 any)
 The functions that have a named argument (v in sort, or sep in cat_sep)
 have those arguments described in the online documentation, but we
 cannot see them in the function prototype (only va_args is shown, 
 which isn't actually the name of any function argument).



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1101) Merge topic/bernhard/ssl_ciphers_vector

2013-12-05 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1101:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Merge topic/bernhard/ssl_ciphers_vector
 ---

 Key: BIT-1101
 URL: https://bro-tracker.atlassian.net/browse/BIT-1101
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann
 Fix For: 2.3


 topic/bernhard/ssl_ciphers_vector changes ciphers in the ssl_client_hello 
 from a set into a vector. This preserves the ordering of the cipher suites 
 the client sent, allowing e.g. better client fingerprinting.



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1097) Unexpected string indexing behavior

2013-12-05 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1097:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Unexpected string indexing behavior
 ---

 Key: BIT-1097
 URL: https://bro-tracker.atlassian.net/browse/BIT-1097
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: 2.2
Reporter: Robin Sommer

 Playing with string indexing/slicing, I'm seeing some (I think) non-intuitive 
 behavior:
 {code}
 global s = 012345;
 print A;
 print s[1:-1];
 print s[1:-2];
 print s[1:-3];
 print s[1:-4];
 print s[1:-5];
 print s[1:-6];
 print s[1:-7];
 print s[1:-8];
 print s[1:-9];
 print ;
 print B;
 print s[-1:-1];
 print s[-1:-2];
 print s[-1:-3];
 print s[-1:-4];
 {code}
 This prints:
 {code}
 A
 12345
 1234
 123
 12
 1
 12345
 12345
 12345
 B
 5
 5
 5
 {code}
 I would instead have expected:
 (1) A to print empty lines for all cases with the 2nd index = -6?
 (2) B to print empty lines for all cases with the 2nd index = -2?
 So, is this intentional? 



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1100) topic/jsiwek/broccoli-vectors

2013-12-05 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1100:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/jsiwek/broccoli-vectors
 -

 Key: BIT-1100
 URL: https://bro-tracker.atlassian.net/browse/BIT-1100
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro, Broccoli
Affects Versions: git/master
Reporter: Jon Siwek
 Fix For: 2.3


 This branch is in the bro and broccoli repos and adds support for broccoli 
 clients to receive events that have arguments w/ vector values.
 Sending events that have arguments w/ vector values is still unsupported.  
 (Broccoli generally seems to be limited in the complexity of types it can 
 create compared to Bro).



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Broccoli and vectors

2013-12-05 Thread Robin Sommer

Half of the support for vectors is there since yesterday:


https://github.com/bro/broccoli/commit/756a8a733b1f03b94afcbb93807813a89b3cfb89

However it sounds like you need the opposite: there's no support yet
for producing events with vectors.

Robin

On Thu, Dec 05, 2013 at 12:54 -0500, you wrote:

 Hi,
 
 I'm implementing an application that sends DNS::Info records via
 Broccoli to Bro.  However, it appears that Broccoli does not fully
 support vectors.  Is this correct?  If it does, can somebody point me
 to an example on how to populate a vector using the Broccoli C API.  I
 searched through the Broccoli docs but could not find anything.
 
 Thanks,
 
 -- Randy
 
 
 
 
 
 
 
 
 
 
 ___
 bro-dev mailing list
 bro-dev@bro.org
 http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
 


-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Bro-Commits] [git/broccoli] topic/jsiwek/broccoli-vectors: Add support for consuming events w/ vector args. (de39868)

2013-12-04 Thread Robin Sommer



On Tue, Dec 03, 2013 at 18:09 +, you wrote:

 and not fixing this could be a common pitfall for users.

Ack, makes sense.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Proposed IOSource reorg

2013-12-04 Thread Robin Sommer



On Tue, Dec 03, 2013 at 18:40 +, you wrote:

 Maybe best would be if the remote serializer code is refactored so the
 code that implements the IOSource interface lives in the iosource/
 tree, while the code that implements Serializer interface lives in a
 separate serializer/ tree?

Could be an option, though I'm not immediately sure how well it would
split.

But one step at a time sounds good in any case, so I'll go ahead with
that and we can later see.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Fwd: [REL - 10amd64-default][security/bro] Failed for bro-2.2 in build

2013-12-03 Thread Robin Sommer

Which clang version is this? I've tried it with a recent version of
the clang 3.4 release branch, and that works fine for me.

But based on the error message, I'm attaching a patch; does that help
by any chance?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
diff --git a/src/logging/writers/SQLite.cc b/src/logging/writers/SQLite.cc
index 46d1f17..25f5cb0 100644
--- a/src/logging/writers/SQLite.cc
+++ b/src/logging/writers/SQLite.cc
@@ -126,7 +126,7 @@ bool SQLite::DoInit(const WriterInfo info, int 
arg_num_fields,
fullpath.append(.sqlite);
string tablename;
 
-   mapconst char*, const char*::const_iterator it = 
info.config.find(tablename);
+   WriterInfo::config_map::const_iterator it = 
info.config.find(tablename);
if ( it == info.config.end() )
{
MsgThread::Info(Fmt(tablename configuration option not found. 
Defaulting to path %s, info.path));

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1097) Unexpected string indexing behavior

2013-11-25 Thread Robin Sommer (JIRA)

Robin Sommer created BIT-1097:
-

 Summary: Unexpected string indexing behavior
 Key: BIT-1097
 URL: https://bro-tracker.atlassian.net/browse/BIT-1097
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: 2.2
Reporter: Robin Sommer


Playing with string indexing/slicing, I'm seeing some (I think) non-intuitive 
behavior:

{code}
global s = 012345;

print A;
print s[1:-1];
print s[1:-2];
print s[1:-3];
print s[1:-4];
print s[1:-5];
print s[1:-6];
print s[1:-7];
print s[1:-8];
print s[1:-9];

print ;

print B;
print s[-1:-1];
print s[-1:-2];
print s[-1:-3];
print s[-1:-4];
{code}

This prints:

{code}
A
12345
1234
123
12
1

12345
12345
12345

B
5

5
5
{code}

I would instead have expected:

(1) A to print empty lines for all cases with the 2nd index = -6?

(2) B to print empty lines for all cases with the 2nd index = -2?

So, is this intentional? 



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Bare Mode

2013-11-22 Thread Robin Sommer



On Fri, Nov 22, 2013 at 15:38 +, you wrote:

 The intention for mode is to allow users more choice in what
 script-level functionality to load.  In practice, I don’t know how
 often it’s used for that.

I'll add that bare mode is essentially what used to be the default
configuration in Bro 2.0. So it's also a way to get back to the old
approach where you would add things as you need them. Bro is more
difficult to use that way but it can reduce resource usage quite a bit
if one really only needs a couple pieces.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1095) Meta ticker tracking patches for potential 2.2.1

Robin Sommer created BIT-1095:
-

 Summary: Meta ticker tracking patches for potential 2.2.1
 Key: BIT-1095
 URL: https://bro-tracker.atlassian.net/browse/BIT-1095
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: 2.2
Reporter: Robin Sommer


I'm creating this ticket to track commit that we would want to back port to 2.2 
if ended up doing a bug fix release 2.2.1



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1095) Meta ticker tracking patches for potential 2.2.1


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1095:
--

Description: I'm creating this ticket to track commits that we would want 
to back port to 2.2 if ended up doing a bug fix release 2.2.1  (was: I'm 
creating this ticket to track commit that we would want to back port to 2.2 if 
ended up doing a bug fix release 2.2.1)

 Meta ticker tracking patches for potential 2.2.1
 

 Key: BIT-1095
 URL: https://bro-tracker.atlassian.net/browse/BIT-1095
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: 2.2
Reporter: Robin Sommer

 I'm creating this ticket to track commits that we would want to back port to 
 2.2 if ended up doing a bug fix release 2.2.1



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1095) Meta ticker tracking patches for potential 2.2.1


[ 
https://bro-tracker.atlassian.net/browse/BIT-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701#comment-14701
 ] 

Robin Sommer commented on BIT-1095:
---

Add to 2.2.1

 Meta ticker tracking patches for potential 2.2.1
 

 Key: BIT-1095
 URL: https://bro-tracker.atlassian.net/browse/BIT-1095
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: 2.2
Reporter: Robin Sommer

 I'm creating this ticket to track commits that we would want to back port to 
 2.2 if ended up doing a bug fix release 2.2.1



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1094) Segmentation Fault in SQLite Writer


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1094:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Segmentation Fault in SQLite Writer
 ---

 Key: BIT-1094
 URL: https://bro-tracker.atlassian.net/browse/BIT-1094
 Project: Bro Issue Tracker
  Issue Type: Patch
  Components: Bro
Affects Versions: 2.2
 Environment: N/A
Reporter: Jon Crussell
Assignee: Bernhard Amann
 Attachments: 0001-Fixed-Segmentation-fault-in-SQLite-Writer.patch


 There is a bug in the SQLite Writer that causes a segmentation fault if the 
 field type is TYPE_TABLE or TYPE_VECTOR. The fix is pretty minor, see 
 attached patch.
 Also available here:
 https://github.com/jcrussell/bro/tree/topic/jcrussell/sqlite-writer-fix



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1093) topic/jsiwek/thread-termination

2013-10-30 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14500#comment-14500
 ] 

Robin Sommer commented on BIT-1093:
---

I looked up when the original {{ ! Killed()}} code got introduced, that was 
in 743fc1680dc9d4c04f38ca80c7ef4e5b88e8f4cb and the commit message points to 
BIT-858. Can you take a look and double-check that the problem described there 
is still addressed with the new version to be sure we don't introduce a 
regression? (Not immediately sure if we have a test that covers that).

 topic/jsiwek/thread-termination
 ---

 Key: BIT-1093
 URL: https://bro-tracker.atlassian.net/browse/BIT-1093
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: git/master
Reporter: Jon Siwek
Assignee: Robin Sommer
 Fix For: 2.2


 The change in this branch should fix the case where the last remaining 
 done/killed thread never got processed (main thread never received pending 
 messages from it or joined/deleted it) until Bro terminates.  Which was 
 problematic if the termination condition depended on processing messages from 
 the last remaining thread.
 The new code's logic is contrary to what it used to be, but I can't figure 
 out what the old was trying to accomplish and think it could only have caused 
 problems.



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1093) topic/jsiwek/thread-termination

2013-10-30 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1093:
--

Status: Reopened  (was: Closed)

 topic/jsiwek/thread-termination
 ---

 Key: BIT-1093
 URL: https://bro-tracker.atlassian.net/browse/BIT-1093
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: git/master
Reporter: Jon Siwek
Assignee: Robin Sommer
 Fix For: 2.2


 The change in this branch should fix the case where the last remaining 
 done/killed thread never got processed (main thread never received pending 
 messages from it or joined/deleted it) until Bro terminates.  Which was 
 problematic if the termination condition depended on processing messages from 
 the last remaining thread.
 The new code's logic is contrary to what it used to be, but I can't figure 
 out what the old was trying to accomplish and think it could only have caused 
 problems.



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1091) Broctl config.py handling of [manager] header is brittle

2013-10-21 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1091:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Broctl config.py handling of [manager] header is brittle
 

 Key: BIT-1091
 URL: https://bro-tracker.atlassian.net/browse/BIT-1091
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
 Environment: RHEL6
Reporter: Bob
  Labels: beta, broctl
 Fix For: 2.2


 $prefix/lib/broctl/BroControl/config.py (line 159, in nodes()) special cases 
 the manager node of the etc/node.cfg config and checks it by the attribute 
 n.name, as opposed to all of the other types that are handled earlier in the 
 function, which get checked by the attribute n.type.  This means that anyone 
 who might try to set a more descriptive manager name, like 
 [broproductionmanager] or [brotestmanager], will break broctl to disastrous 
 effect:
 [root@bro-testmgr bro-2.2-beta]# /opt/bro/bin/broctl install
 removing old policies in /var/bro/spool/installed-scripts-do-not-touch/site 
 ... done.
 removing old policies in /var/bro/spool/installed-scripts-do-not-touch/auto 
 ... done.
 creating policy directories ... done.
 installing site policies ... done.
 generating local-networks.bro ... done.
 Traceback (most recent call last):
   File /opt/bro/bin/broctl, line 980, in module
 loop.onecmd(line)
   File /usr/lib64/python2.6/cmd.py, line 219, in onecmd
 return func(arg)
   File /opt/bro/bin/broctl, line 202, in do_install
 result = install.install(local)
   File /opt/bro/lib/broctl/BroControl/install.py, line 112, in install
 util.force_symlink(manager.cwd(), current)
 AttributeError: 'NoneType' object has no attribute 'cwd'
 abnormal termination, saving state ...
 This should be cleaned up to make this field user-modifiable as the others 
 are, or at the very least we should implement a warning to users that they 
 should not change the name of the field.



--
This message was sent by Atlassian JIRA
(v6.2-OD-01#6204)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] Draft API for new communication library

2013-10-17 Thread Robin Sommer

I have been mulling over how an API for a new communication library
could look like. In short, the idea is to (1) overhaul Bro's current
communication model to make it more flexible and easier to control;
and (2) provide the new functionality in the form of a C library that
replaces Broccoli yet will also be used by Bro itself (i.e., we;ll no
longer have two independent implementations of the same protocol to
maintain).

Draft is here:

http://www.bro.org/development/projects/comm-ng-v2.html

Feedback welcome.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Draft API for new communication library

2013-10-17 Thread Robin Sommer



On Thu, Oct 17, 2013 at 21:54 +, you wrote:

 Would something like Cap'n Proto or Protocol Buffers help in
 defining/maintaining a serialization format?

I didn't know Cap’n Proto so far but I have been wondering about using
Protocol Buffers already as well. We'd have to add another dependency
but it would make this stuff quite a bit less cumbersome. Do you know
if their C version is well maintained? It looks rather old compared to
the standard protobuf distribution. Does Cap'n Proto have a C API?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/dnthayer/doc-changes-for-2.2: Update FreeBSD install instructions (72129ae)

2013-10-16 Thread Robin Sommer


On Mon, Oct 14, 2013 at 15:28 -0700, you wrote:

 Added perl to list of packages to install (it's not installed by default).

What do we require Perl for?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/dnthayer/doc-changes-for-2.2: Update FreeBSD install instructions (72129ae)

2013-10-16 Thread Robin Sommer



On Wed, Oct 16, 2013 at 11:59 -0500, you wrote:

 [ 67%] [Perl] Processing debug commands
 /bin/sh: 1: /usr/bin/perl: not found

Doh! That's unfortunate that a little script like that makes us depend
on Perl. Todo item for 2.3: replace with a Python or awk script.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1089) Please install sample/example broctl .cfg files

2013-10-14 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1089:
--

Status: In Progress  (was: Open)

 Please install sample/example broctl .cfg files
 ---

 Key: BIT-1089
 URL: https://bro-tracker.atlassian.net/browse/BIT-1089
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: BroControl
Reporter: leres
Priority: Low





--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1089) Please install sample/example broctl .cfg files

2013-10-14 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301#comment-14301
 ] 

Robin Sommer commented on BIT-1089:
---

This is now merged into master. Craig, does that solve your problem?



 Please install sample/example broctl .cfg files
 ---

 Key: BIT-1089
 URL: https://bro-tracker.atlassian.net/browse/BIT-1089
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: BroControl
Reporter: leres
Priority: Low





--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1088) pysubnettree-0.20 setup.py has wrong version

2013-10-14 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1088:
--

Resolution: Fixed
Status: Closed  (was: Open)

 pysubnettree-0.20 setup.py has wrong version
 

 Key: BIT-1088
 URL: https://bro-tracker.atlassian.net/browse/BIT-1088
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: pysubnettree
Affects Versions: 2.1
Reporter: Henry Stern
  Labels: setup.py, version

 The 0.20 release of pysubnettree has incorrect data in setup.py.
 setup(name=pysubnettree,
 version=0.19, # Filled in automatically.
 This should read version=0.20 obviously.  It breaks packaging systems like 
 py2dsc.



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/dnthayer/doc-changes-for-2.2: Add README files for most Bro frameworks (60b2c5f)

2013-10-11 Thread Robin Sommer



On Thu, Oct 10, 2013 at 22:29 -0700, Daniel Thayer wrote:

 Add README files for most Bro frameworks

I'm forgetting if it works to put these as comments into the
__load__.bro files? If so, that would be an alternative as it avoids
having a new file in each directory (the README's are easier to find
though when looking at the scripts directly, so I'm a bit torn).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/dnthayer/doc-changes-for-2.2: Add README files for most Bro frameworks (60b2c5f)

2013-10-11 Thread Robin Sommer



On Fri, Oct 11, 2013 at 11:53 -0400, you wrote:

 ultimately all of this is just leading toward creating a more
 formalized module style and having READMEs in the directory would
 probably be good form in general.

Ah, good point.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1045) Review usage of InternalError when parsing network traffic

2013-10-11 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14211#comment-14211
 ] 

Robin Sommer commented on BIT-1045:
---

Going through, I see number of places where I'd argue it's actually a 
programming/logic error that's not something that can be directly/just 
triggered by crafted network traffic. Examples are the RefCnt() checks in 
~ConnectionTimer() and the indent_level check in ODesc. I'm inclined to leave 
them as they were, with the argument being that those kinds of error actually 
*are* best to trigger an abort. E.g, if the reference counting goes awry, 
pretty much all bets are off anyways, and I'd rather have Bro terminate than 
trying to continue. 

So I think the guideline should be avoiding internal errors that happen 
*directly* because of broken network input; not because of (for lack of a 
better term) infrastructure problems in other parts of Bro. (Although I'm 
sure as I go further, I'll find more cases where that definition is  ambiguous  
as well.)

What's your opinion on cases like the above? 

What I could do is go through your diffs and adapt with the above in mind, and 
then we can do another iteration and see if/where we agree. 


 Review usage of InternalError when parsing network traffic
 --

 Key: BIT-1045
 URL: https://bro-tracker.atlassian.net/browse/BIT-1045
 Project: Bro Issue Tracker
  Issue Type: Task
  Components: Bro
Affects Versions: git/master, 2.1
Reporter: Vlad Grigorescu
Assignee: Robin Sommer

 Creating issue for tracking purposes.
 Reporter-InternalError denotes a fatal error, and will cause Bro to stop. 
 Calling this function when parsing network traffic creates the possibility 
 for an attacker using a packet of death, which could stop Bro.
 I suspect that in most cases, a weird should be generated instead, and Bro 
 should just move on to the next packet. A quick grep shows some likely 
 candidates for incorrect use of InternalError:
 src/Sessions.cc:  reporter-InternalError(Bad IP protocol 
 version in DoNextInnerPacket);
 src/Sessions.cc:  reporter-InternalError(fragment block not in 
 dictionary);
 src/Sessions.cc:  reporter-InternalError(fragment block 
 missing);
 src/Sessions.cc:  reporter-InternalError(unknown 
 transport protocol);
 src/Frag.cc:  reporter-InternalError(bad IP version in fragment 
 reassembly);
 src/IP.cc:reporter-InternalError(IPv6_HdrChain::Init with 
 truncated IP header);
 src/IP.cc:reporter-InternalError(IPv6_Hdr_Chain bad 
 header %d, type);
 src/IP.h: reporter-InternalError(bad IP version in 
 IP_Hdr ctor);
 src/RSH.cc:   reporter-InternalError(multiple rsh client names);
 src/RSH.cc:   reporter-InternalError(multiple rsh initial client 
 names);
 src/POP3.cc:  reporter-InternalError(command not known);
 src/Rlogin.cc:reporter-InternalError(multiple rlogin client 
 names);
 src/ICMP.cc:  reporter-InternalError(unexpected IP proto in 
 ICMP analyzer: %d,
 src/ICMP.cc:  reporter-InternalError(unexpected next protocol in 
 ICMP::DeliverPacket());
 src/SMB.cc:   reporter-InternalError(command mismatch for 
 ParseTransaction);
 src/HTTP.cc:  reporter-InternalError(unrecognized HTTP message 
 event);
 src/HTTP.cc:  reporter-InternalError(HTTP ParseRequest failed);
 src/DPM.cc:   reporter-InternalError(unknown protocol);
 src/RPC.cc:   reporter-InternalError(RPC underflow);
 src/RPC.cc:   reporter-InternalError(RPC resync: skipping 
 over data failed);
 src/RPC.cc:   
 reporter-InternalError(inconsistent RPC record marker extraction);



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [JIRA] (BIT-1089) Please install sample/example broctl .cfg files

2013-10-11 Thread Robin Sommer



On Fri, Oct 11, 2013 at 13:37 -0500, you wrote:

 The current behavior where it only installs a .cfg if none exist is
 totally fine. What I'm asking for is that either by default or by
 turning on a cmake argument it would install sample configs.

I'm still not sure I'm really getting the issue but I have an idea:
would a separate make target install-sample-configs work that
unconditionally puts the samples in place? That's something we could
still add to 2.2 even at this point as it doesn't interfere with
anything else.

Robin

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1089) Please install sample/example broctl .cfg files

2013-10-11 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212#comment-14212
 ] 

Robin Sommer commented on BIT-1089:
---





I'm still not sure I'm really getting the issue but I have an idea:
would a separate make target install-sample-configs work that
unconditionally puts the samples in place? That's something we could
still add to 2.2 even at this point as it doesn't interfere with
anything else.

Robin



 Please install sample/example broctl .cfg files
 ---

 Key: BIT-1089
 URL: https://bro-tracker.atlassian.net/browse/BIT-1089
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: BroControl
Reporter: leres
Priority: Low





--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1087) topic/dnthayer/broctl-fixes

2013-10-10 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14203#comment-14203
 ] 

Robin Sommer commented on BIT-1087:
---

Ok, that sounds good. So are these covered by existing tests? Would they catch 
if anything broke?

 topic/dnthayer/broctl-fixes
 ---

 Key: BIT-1087
 URL: https://bro-tracker.atlassian.net/browse/BIT-1087
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch fixes several bugs in broctl:
 1) on Linux, the broctl top command output sometimes shows wrong values for 
 memory statistics,
 2) there is a race condition when the sendmail option is an empty string,
 3) there is a deadlock when broctl runs a local command that produces a 
 sufficiently large amount of output,
 4) the shell scripts used by broctl are not as portable as they could be 
 (specifically, some commands, such as sed, do not support the same options on 
 all implementations)



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1087) topic/dnthayer/broctl-fixes

2013-10-10 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1087:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

.

 topic/dnthayer/broctl-fixes
 ---

 Key: BIT-1087
 URL: https://bro-tracker.atlassian.net/browse/BIT-1087
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch fixes several bugs in broctl:
 1) on Linux, the broctl top command output sometimes shows wrong values for 
 memory statistics,
 2) there is a race condition when the sendmail option is an empty string,
 3) there is a deadlock when broctl runs a local command that produces a 
 sufficiently large amount of output,
 4) the shell scripts used by broctl are not as portable as they could be 
 (specifically, some commands, such as sed, do not support the same options on 
 all implementations)



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1087) topic/dnthayer/broctl-fixes

2013-10-09 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201#comment-14201
 ] 

Robin Sommer commented on BIT-1087:
---

This all makes sense, however some of the fixes make me wary to apply between 
beta and release as I can't really tell by just looking at them if they'll work 
correctly everywhere.  I suppose you have you tested these all on the major 
platforms? 

Does our test suite cover them so that we'd catch if something breaks on one of 
the tested platforms? If/where not, can you add tests that exercise the changed 
code paths (probably not easily possible everywhere, but for some).


 topic/dnthayer/broctl-fixes
 ---

 Key: BIT-1087
 URL: https://bro-tracker.atlassian.net/browse/BIT-1087
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch fixes several bugs in broctl:
 1) on Linux, the broctl top command output sometimes shows wrong values for 
 memory statistics,
 2) there is a race condition when the sendmail option is an empty string,
 3) there is a deadlock when broctl runs a local command that produces a 
 sufficiently large amount of output,
 4) the shell scripts used by broctl are not as portable as they could be 
 (specifically, some commands, such as sed, do not support the same options on 
 all implementations)



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1086) merge topic/bernhard/new-ciphers

2013-10-03 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1086:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 merge topic/bernhard/new-ciphers
 

 Key: BIT-1086
 URL: https://bro-tracker.atlassian.net/browse/BIT-1086
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: Bro
Affects Versions: git/master, 2.2
Reporter: Bernhard Amann
 Fix For: 2.2


 topic/bernhard/new-ciphers adds new ssl ciphers to the constants lists and 
 also adds a few ciphers to the lookup table that were apparently forgotten in 
 the past.



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] doc/install/CHANGES-bro.txt

2013-09-30 Thread Robin Sommer



On Sun, Sep 29, 2013 at 21:25 -0700, you wrote:

 Hmmm, part of the problem is that the top-level CHANGES file has two copies
 of many changes in it.  At line 10466 the changes starting at 2.1-826 repeat.

That must have gotten mixed up at some point. I'll put it on the list
to fix for the release.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] functions truly as globals?

2013-09-26 Thread Robin Sommer



On Thu, Sep 26, 2013 at 16:06 -0400, you wrote:

   some_func = my_func;

Please, no ... That's not only hurting readability profoundly but also
prevents function-level code optimization. Just imagine the impact
once we start compiling scripts ...

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1083) Update scripting documentation

2013-09-23 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1083:
--

Status: Closed  (was: Merge Request)

 Update scripting documentation
 --

 Key: BIT-1083
 URL: https://bro-tracker.atlassian.net/browse/BIT-1083
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: Bro
Reporter: srunnels
Priority: Low
  Labels: documentation,

 Updates based on suggestions by Robin.
 Currently in topic/srunnels/documentation



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1084) topic/dnthayer/broargs

2013-09-23 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1084:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/dnthayer/broargs
 --

 Key: BIT-1084
 URL: https://bro-tracker.atlassian.net/browse/BIT-1084
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch fixes a bug that occurs when someone uses the broargs
 broctl option and it contains a command-line argument with an embedded
 space character.  The scripts that run bro were splitting this argument
 (even if it was correctly quoted in broctl.cfg).  For example, this
 will now work as expected:
 broargs = --filter 'not ip6'



--
This message was sent by Atlassian JIRA
(v6.1-OD-09-WN#6144)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] bro 2.1 vs clang

2013-09-21 Thread Robin Sommer


On Fri, Sep 20, 2013 at 22:11 -0700, you wrote:

 /home/ports/security/bro/work/bro-2.1/src/Expr.cc:2392:9: error:
 reference to 'is_assignable' is ambiguous

clang will be happy if you change it to this:

+   if ( ! ::is_assignable(op-Type()) )

I've actually fixed that in a branch, thanks for reminding me to merge
it in for 2.2. :-)

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1081) topic/jsiwek/raw-exec-pgrp

2013-09-20 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1081:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/jsiwek/raw-exec-pgrp
 --

 Key: BIT-1081
 URL: https://bro-tracker.atlassian.net/browse/BIT-1081
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Affects Versions: git/master
Reporter: Jon Siwek
Assignee: Daniel Thayer
 Fix For: 2.2


 Daniel, can you do a sanity check w/ this branch on your Ubuntu system to 
 confirm it fixes the problem w/ the executestream test leaving behind 'tail' 
 processes?  If it does, you can change this to a merge request.



--
This message was sent by Atlassian JIRA
(v6.1-OD-08#6143)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1072:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 merge topic/bernhard/hyperloglog
 

 Key: BIT-1072
 URL: https://bro-tracker.atlassian.net/browse/BIT-1072
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann
 Fix For: 2.2

 Attachments: out.pdf


 The branch adds support for the hyperloglog data structure.
 In the branch, core/leaks/basic-cluster.bro currently faisl. However, this 
 seems to be unrelated to hll and just to be triggered by the addition of it 
 to the sumstats tests. It looks like some kind of scriptland issue. pprof 
 output is attached. (master, workers don't leak memory)



--
This message was sent by Atlassian JIRA
(v6.1-OD-08#6143)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1078) topic/dnthayer/documentation


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1078:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/dnthayer/documentation
 

 Key: BIT-1078
 URL: https://bro-tracker.atlassian.net/browse/BIT-1078
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: bro-aux
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch updates the documentation for bro-aux.



--
This message was sent by Atlassian JIRA
(v6.1-OD-08#6143)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-950) Add client/server random to SSL hello events


 [ 
https://bro-tracker.atlassian.net/browse/BIT-950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-950:
-

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Add client/server random to SSL hello events
 

 Key: BIT-950
 URL: https://bro-tracker.atlassian.net/browse/BIT-950
 Project: Bro Issue Tracker
  Issue Type: Patch
  Components: Bro
Affects Versions: git/master
Reporter: ewust
Assignee: Bernhard Amann
Priority: Low
 Fix For: 2.2

 Attachments: 0001-Add-client-server-random-to-ssl-hello-events.patch


 ssl_client_hello and ssl_server_hello should provide applications with the 
 nonces (client/server random) in the SSL hello messages. This can be used for 
 steganographic applications, or can be used to detect entropy problems.



--
This message was sent by Atlassian JIRA
(v6.1-OD-08#6143)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1079) topic/dnthayer/compilerwarn


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1079:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/dnthayer/compilerwarn
 ---

 Key: BIT-1079
 URL: https://bro-tracker.atlassian.net/browse/BIT-1079
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: Bro
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch fixes several compiler warnings and one cmake warning.



--
This message was sent by Atlassian JIRA
(v6.1-OD-08#6143)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1074) topic/dnthayer/broctl-tests


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1074:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/dnthayer/broctl-tests
 ---

 Key: BIT-1074
 URL: https://bro-tracker.atlassian.net/browse/BIT-1074
 Project: Bro Issue Tracker
  Issue Type: Improvement
  Components: BroControl
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch adds tests for newer features of broctl (CPU pinning,
 PF_RING multiple cluster IDs, and the env_vars option).



--
This message was sent by Atlassian JIRA
(v6.1-OD-08#6143)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog

2013-08-31 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13820#comment-13820
 ] 

Robin Sommer commented on BIT-1072:
---

I fixed the bug introduced, did some more polishing, and also made the 
confidence a parameter to hll init. 

Merged into master now, but please still work on the Doxygen comments. 

 merge topic/bernhard/hyperloglog
 

 Key: BIT-1072
 URL: https://bro-tracker.atlassian.net/browse/BIT-1072
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann
 Fix For: 2.2

 Attachments: out.pdf


 The branch adds support for the hyperloglog data structure.
 In the branch, core/leaks/basic-cluster.bro currently faisl. However, this 
 seems to be unrelated to hll and just to be triggered by the addition of it 
 to the sumstats tests. It looks like some kind of scriptland issue. pprof 
 output is attached. (master, workers don't leak memory)



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog

2013-08-30 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13817#comment-13817
 ] 

Robin Sommer commented on BIT-1072:
---

I'm getting a number of conflicts when merging into master. Please merge the 
branch with master first.

 merge topic/bernhard/hyperloglog
 

 Key: BIT-1072
 URL: https://bro-tracker.atlassian.net/browse/BIT-1072
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann
 Fix For: 2.2

 Attachments: out.pdf


 The branch adds support for the hyperloglog data structure.
 In the branch, core/leaks/basic-cluster.bro currently faisl. However, this 
 seems to be unrelated to hll and just to be triggered by the addition of it 
 to the sumstats tests. It looks like some kind of scriptland issue. pprof 
 output is attached. (master, workers don't leak memory)



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog

2013-08-30 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1072:
--

Status: Open  (was: Merge Request)

 merge topic/bernhard/hyperloglog
 

 Key: BIT-1072
 URL: https://bro-tracker.atlassian.net/browse/BIT-1072
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann
 Fix For: 2.2

 Attachments: out.pdf


 The branch adds support for the hyperloglog data structure.
 In the branch, core/leaks/basic-cluster.bro currently faisl. However, this 
 seems to be unrelated to hll and just to be triggered by the addition of it 
 to the sumstats tests. It looks like some kind of scriptland issue. pprof 
 output is attached. (master, workers don't leak memory)



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog

2013-08-30 Thread Robin Sommer (JIRA)


[ 
https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13819#comment-13819
 ] 

Robin Sommer commented on BIT-1072:
---

I ended up refactoring and reformatting this quite a bit, it's in 
topic/robin/hyperlolog-merge. However, I broke something, the tests aren't 
working. Need to debug that later. In the meantime, some requests/questions:

- Please look over my changes and see if they make sense. (You don't need to 
track down the bug; I take the blame for that :).

- Can you please rework the Doxygen comments in HyperLogLog.h so that the 
descriptions for the public methods are understandable on their own. Right now 
I can't really follow them as often they talk about internal 
parameters/functionality. What you could do is provide a short overview of the 
data structure parameters in the class' doc string, and then refer to that in 
the methods. Also, please use the @param and @return syntax. (Start from my 
branch with this: I already reformatted and reordered things there quite a bit.)

- I don't understand what can be parameterized by the user and what not (and 
why not). One can give an error margin to the actor, but the confidence is a 
compile time constant. Also, where are the magic alpha_m values in *.cc coming 
from? Are these indeed always static values that don't depend on any parameters?



 merge topic/bernhard/hyperloglog
 

 Key: BIT-1072
 URL: https://bro-tracker.atlassian.net/browse/BIT-1072
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: Bernhard Amann
 Fix For: 2.2

 Attachments: out.pdf


 The branch adds support for the hyperloglog data structure.
 In the branch, core/leaks/basic-cluster.bro currently faisl. However, this 
 seems to be unrelated to hll and just to be triggered by the addition of it 
 to the sumstats tests. It looks like some kind of scriptland issue. pprof 
 output is attached. (master, workers don't leak memory)



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1070) topic/dnthayer/bug-fixes

2013-08-28 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1070:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 topic/dnthayer/bug-fixes
 

 Key: BIT-1070
 URL: https://bro-tracker.atlassian.net/browse/BIT-1070
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BTest
Reporter: Daniel Thayer
 Fix For: 2.2


 This branch contains some fixes to btest and the README.



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit

2013-08-28 Thread Robin Sommer (JIRA)


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1016:
--

Resolution: Merged  (was: Fixed)
Status: Closed  (was: Merge Request)

 Option to extend uids to 128 bit
 

 Key: BIT-1016
 URL: https://bro-tracker.atlassian.net/browse/BIT-1016
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: rhave
Assignee: Jon Siwek
Priority: Low
 Fix For: 2.2


 Bro's uids are currently 64 bits, which makes them collide with a 50% chance 
 after 5.1 x 10^9^ different uids (see 
 http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
 I'm currently generating uuids of 128 bit to replace the native uids in bro, 
 as I'm using them as keys in a database, but this requires rewriting of the 
 bro-logs. I suspect that more people could benefit from an option to extend 
 the uids to 128 bit.
 I've made a quick and dirty patch to change most of the uids to 128 bit 
 (file_analysis uids are missing). The patch is ugly, and is only to show some 
 of the functionality I would like: http://pastebin.com/GkaGejNc



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit


 [ 
https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robin Sommer updated BIT-1016:
--

Status: Open  (was: Merge Request)

 Option to extend uids to 128 bit
 

 Key: BIT-1016
 URL: https://bro-tracker.atlassian.net/browse/BIT-1016
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: rhave
Assignee: Jon Siwek
Priority: Low
 Fix For: 2.2


 Bro's uids are currently 64 bits, which makes them collide with a 50% chance 
 after 5.1 x 10^9^ different uids (see 
 http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
 I'm currently generating uuids of 128 bit to replace the native uids in bro, 
 as I'm using them as keys in a database, but this requires rewriting of the 
 bro-logs. I suspect that more people could benefit from an option to extend 
 the uids to 128 bit.
 I've made a quick and dirty patch to change most of the uids to 128 bit 
 (file_analysis uids are missing). The patch is ugly, and is only to show some 
 of the functionality I would like: http://pastebin.com/GkaGejNc



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message


[ 
https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806#comment-13806
 ] 

Robin Sommer commented on BIT-1068:
---

Is the true here intentional?

{code}
[…]
# message just in case there's some other reason for the failure).
true
if [ $? -eq 0 ]; then
[…]
{code}



 pin_cpus error message
 --

 Key: BIT-1068
 URL: https://bro-tracker.atlassian.net/browse/BIT-1068
 Project: Bro Issue Tracker
  Issue Type: Problem
  Components: BroControl
Affects Versions: 2.2
Reporter: Seth Hall
Assignee: Daniel Thayer
 Fix For: 2.2


 I seem to be having a problem with the cpu_pin feature of broctl.  I'm 
 getting the following output...
 [rootsh@xx worker-1-6]# cat stderr.log 
 sched_setaffinity: Invalid argument
 failed to set pid 0's affinity.
 Daniel, any clue what I should be looking into or information I can provide?



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit


[ 
https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808#comment-13808
 ] 

Robin Sommer commented on BIT-1016:
---





Good point.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin


 Option to extend uids to 128 bit
 

 Key: BIT-1016
 URL: https://bro-tracker.atlassian.net/browse/BIT-1016
 Project: Bro Issue Tracker
  Issue Type: New Feature
  Components: Bro
Affects Versions: git/master
Reporter: rhave
Assignee: Jon Siwek
Priority: Low
 Fix For: 2.2


 Bro's uids are currently 64 bits, which makes them collide with a 50% chance 
 after 5.1 x 10^9^ different uids (see 
 http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
 I'm currently generating uuids of 128 bit to replace the native uids in bro, 
 as I'm using them as keys in a database, but this requires rewriting of the 
 bro-logs. I suspect that more people could benefit from an option to extend 
 the uids to 128 bit.
 I've made a quick and dirty patch to change most of the uids to 128 bit 
 (file_analysis uids are missing). The patch is ugly, and is only to show some 
 of the functionality I would like: http://pastebin.com/GkaGejNc



--
This message was sent by Atlassian JIRA
(v6.1-OD-06#6139)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit

2013-08-27 Thread Robin Sommer



On Tue, Aug 27, 2013 at 20:35 +, you wrote:

 FWIW, I prefer Chex for the simple reason that if I double-click it,
 it selects the whole uid (including the C), and I can then copy the
 whole thing.

Good point.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org
ICSI/LBNL* Fax   +1 (510) 666-2956 * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit

2013-08-27 Thread Robin Sommer

I like that idea. Storage-wise it's 2x64 bit anyways.

Robin

On Tue, Aug 27, 2013 at 16:04 -0500, you wrote:

 Regarding performance: another option would be to use 128-bit UUIDs
 internally and just chop of 32 bytes if a 96-bit UUID is desired,
 assuming the bits in the UUID are distributed uniformly. Then we could
 use a fixed-size array and just change how the data is interpreted at
 script land.



  Option to extend uids to 128 bit
  
 
  Key: BIT-1016
  URL: https://bro-tracker.atlassian.net/browse/BIT-1016
  Project: Bro Issue Tracker
   Issue Type: New Feature
   Components: Bro
 Affects Versions: git/master
 Reporter: rhave
 Assignee: Jon Siwek
 Priority: Low
  Fix For: 2.2
 
 
  Bro's uids are currently 64 bits, which makes them collide with a 50% 
  chance after 5.1 x 10^9^ different uids (see 
  http://en.wikipedia.org/wiki/Birthday_problem#Probability_table).
  I'm currently generating uuids of 128 bit to replace the native uids in 
  bro, as I'm using them as keys in a database, but this requires rewriting 
  of the bro-logs. I suspect that more people could benefit from an option to 
  extend the uids to 128 bit.
  I've made a quick and dirty patch to change most of the uids to 128 bit 
  (file_analysis uids are missing). The patch is ugly, and is only to show 
  some of the functionality I would like: http://pastebin.com/GkaGejNc
 
 
 
 --
 This message was sent by Atlassian JIRA
 (v6.1-OD-06#6139)
 ___
 bro-dev mailing list
 bro-dev@bro.org
 http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
 


___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit