[Bro-Dev] [JIRA] (BIT-1046) topic/jsiwek/exec-module
[ https://bro-tracker.atlassian.net/browse/BIT-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1046: --- Status: Merge Request (was: Open) topic/jsiwek/exec-module Key: BIT-1046 URL: https://bro-tracker.atlassian.net/browse/BIT-1046 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.1 Reporter: Jon Siwek Fix For: 2.2 Some scripts for executing system commands and getting the results (stderr/stdout, exit code, file output) back in to Bro. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] [JIRA] (BIT-1043) LRU Table implementation
Fix Version is what's used to put something on the roadmap. i.e. it can be
used like milestones were w/ Trac. (I think that's what was intended by
Seth's Affects Version change.)
- Jon
On Jul 29, 2013, at 9:46 AM, Jon Siwek (JIRA) j...@bro-tracker.atlassian.net
wrote:
[
https://bro-tracker.atlassian.net/browse/BIT-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Siwek updated BIT-1043:
---
Fix Version/s: 2.3
LRU Table implementation
Key: BIT-1043
URL: https://bro-tracker.atlassian.net/browse/BIT-1043
Project: Bro Issue Tracker
Issue Type: Improvement
Components: Bro
Affects Versions: 2.1
Reporter: Jordi Ros-Giralt
Fix For: 2.3
Attaching below the email description i exchanged with Seth and Robin
describing this work.
--
Hi Seth and Robin,
We got the repo up, you can get to our branch as follows:
git clone --recursive https://github.com/giralt/bro.git
cd bro/
git checkout lru-table
We would be happy to contribute this code to the Bro community. This is
what it does:
- It implements LRU tables for Bro
- A Bro table can be enhanced with the LRU functionality with the following
new table attributes:
lru_table: enhance the table with LRU functionality
size_limit=n: if adding an element to the table makes the size of the table
larger than n, then drop the LRU element from that table before inserting
the new element. n=0 means table size can be infinite (so don't drop
elements from it)
drop_func=callback_func: defines a programmable callback function that gets
called automatically every time an element from the LRU table is dropped due
to hitting the size_limit. The prototype of this callback must be as
follows:
function callback_func(t: table[keytype] of valuetype, key: keytype, val:
valuetype): count
- It adds the following bif functions:
function get_lru%(v: any%): any
function get_mru%(v: any%): any
function get_lru_key%(v: any%): any
function get_mru_key%(v: any%): any
- Example:
function freed(t: table[port] of string, key: port, val: string): count {
print Dropped; }
local port_names: table[port] of string lru size_limit=2 drop_func=freed;
In terms of applications, we are currently using this feature for the
chimera-to-bro compiler we are working on:
http://www.chimera-query.org/index.html
We thought that we could also use this feature to provide a sort of memory
management facility for Bro. I had a talk with Seth some weeks ago about
this. Something like the LRU implementation allows programmers to put
bounds on the size of tables and prioritize which elements can be dropped
first upon memory exhaustion scenarios. Perhaps an idea here would be to
develop a garbage collector (could be done using Bro language itself,
perhaps as a framework) which would be run upon hitting a certain memory
usage watermark and which would mainly run through the set of tables marked
as garbage collectable dropping LRU elements from them to help
reduce/eliminate the risk of running out of memory.
Should this be something interesting, what are the steps we would need to do
to open source the LRU code into Bro?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
administrators:
https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/exec-module: Exec module changes/fixes. (73eb87a)
Exec module changes/fixes. Do you feel comfortable with this being merged into master now? I'm not aware of any outstanding problems w/ it, so yes. I made a merge request ticket. - Jon ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1046) topic/jsiwek/exec-module
Jon Siwek created BIT-1046: -- Summary: topic/jsiwek/exec-module Key: BIT-1046 URL: https://bro-tracker.atlassian.net/browse/BIT-1046 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.1 Reporter: Jon Siwek Fix For: 2.2 Some scripts for executing system commands and getting the results (stderr/stdout, exit code, file output) back in to Bro. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/exec-module: Exec module changes/fixes. (73eb87a)
On Jul 29, 2013, at 10:56 AM, Siwek, Jonathan Luke jsi...@illinois.edu wrote: Exec module changes/fixes. Do you feel comfortable with this being merged into master now? I'm not aware of any outstanding problems w/ it, so yes. I made a merge request ticket. Cool, thanks. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1045) Review usage of InternalError when parsing network traffic
[ https://bro-tracker.atlassian.net/browse/BIT-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400#comment-13400 ] Robin Sommer commented on BIT-1045: --- Ack, InternalError() is not something that external input should be able to trigger. I already removed a number of these over time, but never looked systematically for them. Agreed, though sometimes they aren't about the traffic but about a logic error in decoding it; it would be good to still differentiate those cases from a broken packet, however indeed without aborting. Review usage of InternalError when parsing network traffic -- Key: BIT-1045 URL: https://bro-tracker.atlassian.net/browse/BIT-1045 Project: Bro Issue Tracker Issue Type: Task Components: Bro Affects Versions: git/master, 2.1 Reporter: Vlad Grigorescu Creating issue for tracking purposes. Reporter-InternalError denotes a fatal error, and will cause Bro to stop. Calling this function when parsing network traffic creates the possibility for an attacker using a packet of death, which could stop Bro. I suspect that in most cases, a weird should be generated instead, and Bro should just move on to the next packet. A quick grep shows some likely candidates for incorrect use of InternalError: src/Sessions.cc: reporter-InternalError(Bad IP protocol version in DoNextInnerPacket); src/Sessions.cc: reporter-InternalError(fragment block not in dictionary); src/Sessions.cc: reporter-InternalError(fragment block missing); src/Sessions.cc: reporter-InternalError(unknown transport protocol); src/Frag.cc: reporter-InternalError(bad IP version in fragment reassembly); src/IP.cc:reporter-InternalError(IPv6_HdrChain::Init with truncated IP header); src/IP.cc:reporter-InternalError(IPv6_Hdr_Chain bad header %d, type); src/IP.h: reporter-InternalError(bad IP version in IP_Hdr ctor); src/RSH.cc: reporter-InternalError(multiple rsh client names); src/RSH.cc: reporter-InternalError(multiple rsh initial client names); src/POP3.cc: reporter-InternalError(command not known); src/Rlogin.cc:reporter-InternalError(multiple rlogin client names); src/ICMP.cc: reporter-InternalError(unexpected IP proto in ICMP analyzer: %d, src/ICMP.cc: reporter-InternalError(unexpected next protocol in ICMP::DeliverPacket()); src/SMB.cc: reporter-InternalError(command mismatch for ParseTransaction); src/HTTP.cc: reporter-InternalError(unrecognized HTTP message event); src/HTTP.cc: reporter-InternalError(HTTP ParseRequest failed); src/DPM.cc: reporter-InternalError(unknown protocol); src/RPC.cc: reporter-InternalError(RPC underflow); src/RPC.cc: reporter-InternalError(RPC resync: skipping over data failed); src/RPC.cc: reporter-InternalError(inconsistent RPC record marker extraction); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1045) Review usage of InternalError when parsing network traffic
[ https://bro-tracker.atlassian.net/browse/BIT-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401#comment-13401 ] Vern Paxson commented on BIT-1045: -- In line with what you frame, the history behind these is that they're meant to reflect should-never-happen situations; not weird activity, but apparent internal inconsistencies in Bro's processing/execution. So they don't really fit with the notion of weird. (Of course it's definitely possible there's some mission-creep and InternalError is misused when Weird really is the right notion.) That said, for sure I agree that we don't want to give adversaries a way to tickle a Bro crash. So ideally the solution here would be to come up with something similar to the weird/notice framework, but that expicitly captures the notion that Bro-is-confused rather than something-happened-on-the-network. Vern Review usage of InternalError when parsing network traffic -- Key: BIT-1045 URL: https://bro-tracker.atlassian.net/browse/BIT-1045 Project: Bro Issue Tracker Issue Type: Task Components: Bro Affects Versions: git/master, 2.1 Reporter: Vlad Grigorescu Creating issue for tracking purposes. Reporter-InternalError denotes a fatal error, and will cause Bro to stop. Calling this function when parsing network traffic creates the possibility for an attacker using a packet of death, which could stop Bro. I suspect that in most cases, a weird should be generated instead, and Bro should just move on to the next packet. A quick grep shows some likely candidates for incorrect use of InternalError: src/Sessions.cc: reporter-InternalError(Bad IP protocol version in DoNextInnerPacket); src/Sessions.cc: reporter-InternalError(fragment block not in dictionary); src/Sessions.cc: reporter-InternalError(fragment block missing); src/Sessions.cc: reporter-InternalError(unknown transport protocol); src/Frag.cc: reporter-InternalError(bad IP version in fragment reassembly); src/IP.cc:reporter-InternalError(IPv6_HdrChain::Init with truncated IP header); src/IP.cc:reporter-InternalError(IPv6_Hdr_Chain bad header %d, type); src/IP.h: reporter-InternalError(bad IP version in IP_Hdr ctor); src/RSH.cc: reporter-InternalError(multiple rsh client names); src/RSH.cc: reporter-InternalError(multiple rsh initial client names); src/POP3.cc: reporter-InternalError(command not known); src/Rlogin.cc:reporter-InternalError(multiple rlogin client names); src/ICMP.cc: reporter-InternalError(unexpected IP proto in ICMP analyzer: %d, src/ICMP.cc: reporter-InternalError(unexpected next protocol in ICMP::DeliverPacket()); src/SMB.cc: reporter-InternalError(command mismatch for ParseTransaction); src/HTTP.cc: reporter-InternalError(unrecognized HTTP message event); src/HTTP.cc: reporter-InternalError(HTTP ParseRequest failed); src/DPM.cc: reporter-InternalError(unknown protocol); src/RPC.cc: reporter-InternalError(RPC underflow); src/RPC.cc: reporter-InternalError(RPC resync: skipping over data failed); src/RPC.cc: reporter-InternalError(inconsistent RPC record marker extraction); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] viability of stand-alone (not installed) Bro
I don't think that is intended behavior, but rather an unintended consequence of some of the work on the file analysis framework and shipping with our own magic DB. Perhaps Jon can elaborate more on what it would take to fix this? On Jul 29, 2013, at 10:09 PM, Vern Paxson v...@icir.org wrote: For various reasons I sometimes like to run Bro out of the directory I used to build it, rather than installing it. With a fresh git pull, when doing this I get: % build/src/bro internal error: can't load magic file /usr/local/bro/share/bro/magic: could not find any valid magic files! Abort Well harumph. However, the reason for this note (rather than a shiny-new-tracker ticket) is I'm wondering whether filing the ticket is a lost cause (i.e., the current philosophy is it's okay if things only work post make install) ... ? Vern ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1044) topic/seth/faf-updates ready for merge
[ https://bro-tracker.atlassian.net/browse/BIT-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1044: -- Resolution: Merged Status: Closed (was: Merge Request) topic/seth/faf-updates ready for merge -- Key: BIT-1044 URL: https://bro-tracker.atlassian.net/browse/BIT-1044 Project: Bro Issue Tracker Issue Type: Task Reporter: Seth Hall Assignee: Robin Sommer Fix For: 2.2 Big updates to logs and functionality of the files framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] viability of stand-alone (not installed) Bro
Try source build/bro-path-dev.sh. Cool, that does it. The only problem is surely I'm going to fail to remember that bit of voodoo, and bug you with similar questions in the future at least three more times :-P. Vern ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] viability of stand-alone (not installed) Bro
On Mon, Jul 29, 2013 at 20:08 -0700, you wrote: I'm wondering whether filing the ticket is a lost cause (i.e., the current philosophy is it's okay if things only work post make install) ... ? Quite the opposite: I'm sure most of us run it right out of the source tree more often than not. Try source build/bro-path-dev.sh. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1046) topic/jsiwek/exec-module
[ https://bro-tracker.atlassian.net/browse/BIT-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1046: -- Resolution: Merged Status: Closed (was: Merge Request) topic/jsiwek/exec-module Key: BIT-1046 URL: https://bro-tracker.atlassian.net/browse/BIT-1046 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.1 Reporter: Jon Siwek Fix For: 2.2 Some scripts for executing system commands and getting the results (stderr/stdout, exit code, file output) back in to Bro. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
