Re: [Bro-Dev] Geo Location Plugin
On Thu, Oct 16, 2014 at 11:54 -0700, you wrote: That makes sense. The next logical question would be, what makes a plugin special enough to be included the core? I think GeoIP, being legacy like Robin mentioned, qualifies. Yeah, as Seth writes, it'll be a fine line. The other thing is infrastructure: right now we have one bro-plugins repository, which is pulled into aux/ but nothing in there is built automatically. I see three options: (1) Leave as it is; i.e., not build any plugins by default. (2) Select some plugins in bro-plugins that build automatically with Bro. (3) Split bro-plugins into two repositories. One set would build with Bro, the other not. We'd need to decide if both would be pulled into the Bro source tree, or not. Sounds we don't want (1), but I'm undecided between (2) and (3). Opinions? For both (2) and (3) we'll need to integrate the selected plugins into Bro's build process, including passing through configure options. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 17, 2014, at 8:16 AM, Robin Sommer ro...@icir.org wrote: (2) Select some plugins in bro-plugins that build automatically with Bro. (3) Split bro-plugins into two repositories. One set would build with Bro, the other not. We'd need to decide if both would be pulled into the Bro source tree, or not. Sounds we don't want (1), but I'm undecided between (2) and (3). Opinions? Maybe a minor downside to (3) would be it’s less smooth of a transition if you ever wanted to promote/demote a plugin — involves moving code across repo as well as changing configure/CMake glue whereas (2) I’m guessing just involves the later with plugin code staying unchanged/unmoved. Maybe try approach (2) first as it seems simpler, then find out along the way what improvements are needed. - Jon ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Fri, Oct 17, 2014 at 15:04 +, you wrote: Maybe try approach (2) first as it seems simpler, then find out along the way what improvements are needed. Yeah, I can see that. I wondering if we could find a mechanism to automatically hook selected plugins into the Bro build without changing anythign there. Maybe they'd ship with a configure hook that the main configure searches all plugins for; or something to that effect. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 17, 2014, at 11:14 AM, Robin Sommer ro...@icir.org wrote: Maybe they'd ship with a configure hook that the main configure searches all plugins for; or something to that effect. What I don't like about this is that eventually these should all be pushed into external plugins so I feel like we should just keep the amount of work minimal for now. We're just fighting against the non-existence of a package manager right now. :) I like plan #2 the best for now too. Although I guess without doing the configure option thing, it would turn things like geoip into a required dependency. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
I think that is reasonable for some things. On Oct 16, 2014, at 8:02 AM, Robin Sommer ro...@icir.org wrote: (unless we decide to build (some) plugins by default, which currently isn't happening). -- Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity Directorate National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure. ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
I think having some plugins distributed and built with Bro is reasonable, too. As long as functionality and interfaces don't change current users shouldn't notice a difference. How eventually moving the plugin to a package manager will affect users is something else to consider. -AK On Oct 16, 2014 6:21 AM, Slagell, Adam J slag...@illinois.edu wrote: I think that is reasonable for some things. On Oct 16, 2014, at 8:02 AM, Robin Sommer ro...@icir.org wrote: (unless we decide to build (some) plugins by default, which currently isn't happening). -- Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity Directorate National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure. ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
+1 that core plugins be built / distributed with bro. Re: package / plugin management, coming from a Java EE background, one of the things I don't like about some of the OSGi software (read: highly modular, plugin-driven) I've worked with is how confusing it can get to track versions of each specific bundle / plugin to know how / when to update them. Binding a specific set of core plugin versions to application releases mitigates some of that complexity in my experience. In other words, I'd be a bit hesitant to agree that bro should individually package *core* plugins and offer them independently through a repo, just because it will probably require a great deal of care to update those (since there could be e.g. side effects / dependencies that aren't immediately obvious). An external repository / package manager seems like a good fit for non-core plugins, though. -Gilbert On Oct 16, 2014 11:41 AM, anthony kasza anthony.ka...@gmail.com wrote: I think having some plugins distributed and built with Bro is reasonable, too. As long as functionality and interfaces don't change current users shouldn't notice a difference. How eventually moving the plugin to a package manager will affect users is something else to consider. -AK On Oct 16, 2014 6:21 AM, Slagell, Adam J slag...@illinois.edumailto:slag...@illinois.edu wrote: I think that is reasonable for some things. On Oct 16, 2014, at 8:02 AM, Robin Sommer ro...@icir.orgmailto:ro...@icir.org wrote: (unless we decide to build (some) plugins by default, which currently isn't happening). -- Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity Directorate National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.infohttp://www.slagell.info Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure. ___ bro-dev mailing list bro-dev@bro.orgmailto:bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
That makes sense. The next logical question would be, what makes a plugin special enough to be included the core? I think GeoIP, being legacy like Robin mentioned, qualifies. -AK On Oct 16, 2014 9:21 AM, Clark, Gilbert gc355...@ohio.edu wrote: +1 that core plugins be built / distributed with bro. Re: package / plugin management, coming from a Java EE background, one of the things I don't like about some of the OSGi software (read: highly modular, plugin-driven) I've worked with is how confusing it can get to track versions of each specific bundle / plugin to know how / when to update them. Binding a specific set of core plugin versions to application releases mitigates some of that complexity in my experience. In other words, I'd be a bit hesitant to agree that bro should individually package *core* plugins and offer them independently through a repo, just because it will probably require a great deal of care to update those (since there could be e.g. side effects / dependencies that aren't immediately obvious). An external repository / package manager seems like a good fit for non-core plugins, though. -Gilbert On Oct 16, 2014 11:41 AM, anthony kasza anthony.ka...@gmail.com wrote: I think having some plugins distributed and built with Bro is reasonable, too. As long as functionality and interfaces don't change current users shouldn't notice a difference. How eventually moving the plugin to a package manager will affect users is something else to consider. -AK On Oct 16, 2014 6:21 AM, Slagell, Adam J slag...@illinois.edu wrote: I think that is reasonable for some things. On Oct 16, 2014, at 8:02 AM, Robin Sommer ro...@icir.org wrote: (unless we decide to build (some) plugins by default, which currently isn't happening). -- Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity Directorate National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure. ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 16, 2014, at 2:54 PM, anthony kasza anthony.ka...@gmail.com wrote: That makes sense. The next logical question would be, what makes a plugin special enough to be included the core? I think GeoIP, being legacy like Robin mentioned, qualifies. Yep. This is going to just remain as-always a careful balance between providing extra functionality and keeping the number of dependencies low for Bro. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 14, 2014, at 7:48 PM, anthony kasza anthony.ka...@gmail.com wrote: From what I can tell, I don't have geo_location = internal_type(geo_location)-AsRecordType(); in the right location. This line is from the init_net_var() function from NetVar.cc, which gets called by main.cc. Maybe that can just be completely removed if Bro proper no longer relies on that type since all the related functionality is now provided by the plugin? everything sitting in a single .bif file. Would it be useful to post that? How about putting the entire plugin source directory in a github repo? That should make it easy for others to start poking at the same code as you. - Jon ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 15, 2014, at 10:46 AM, Siwek, Jon jsi...@illinois.edu wrote: Maybe that can just be completely removed if Bro proper no longer relies on that type since all the related functionality is now provided by the plugin? Agreed. Any code related to GeoIP should move into the plugin, including types. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 15, 2014, at 1:22 PM, anthony kasza anthony.ka...@gmail.com wrote: What are the general conventions for doing this? Oh, also: https://www.bro.org/development/howtos/process.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
Thanks all. After moving declarations around, nm shows all my expected symbols to be defined. I'm now receiving internal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: internal type geo_location missing. It seems this error is being caused by the check on the return value of lookup_ID() done within the internal_type() function in Var.cc. From what I can tell, I don't have geo_location = internal_type(geo_location)-AsRecordType(); in the right location. This line is from the init_net_var() function from NetVar.cc, which gets called by main.cc. I thought it might be a clash in name/module spaces and tried using the init-plugin script with some unique values, but I still receive the same error. I'm not sure if it matters but I when I run bro -NNb I can see my inactive dynamic plugin, so I know Bro is aware of it. I currently have everything sitting in a single .bif file. Would it be useful to post that? -AK On Tue, Oct 7, 2014 at 8:25 AM, Robin Sommer ro...@icir.org wrote: On Mon, Oct 06, 2014 at 21:46 -0700, you wrote: Thanks Robin. Everything on the Writing Bro Plugins page is clear although the debugging section seems thin to me. Then again, I've have never compiled Bro with debugging enabled before. Ok, I'll try to extend that a bit more, it's written mainly from the perspective of adding debugging output to your plugin; not in terms of leveraging the existing output to see why something's not working. Are there plans to expand the Types section of that page? I would like to, but it's quite a bit of work to fill in the TODOs, so honestly I don't see that coming very soon. The problem is that most of the missing TODOs are actually not really a matter per se of writing a plugin or not, but require text on how to write the corresponding code for Bro in general (in other words, even before moving to plugins, much of that would have been quite similar). For now, the best way to figure this out is to look at existing code, i.e., for the builtin-in stuff other *.bif files. I think I'm not declaring a scriptland record type correctly. Feel free to post error messages here. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
Hi: A mismatched function definition in .h / implementation in .cc usually causes this issue for me. Maybe try running the mangled symbol that's reported as missing through c++filt to see what it really is (if you haven't already) and use nm or the like to make sure the plugin library file contains that symbol? -Gilbert From: bro-dev-boun...@bro.org [bro-dev-boun...@bro.org] on behalf of anthony kasza [anthony.ka...@gmail.com] Sent: Tuesday, October 07, 2014 12:46 AM To: Robin Sommer Cc: bro-dev@bro.org Subject: Re: [Bro-Dev] Geo Location Plugin Thanks Robin. Everything on the Writing Bro Plugins page is clear although the debugging section seems thin to me. Then again, I've have never compiled Bro with debugging enabled before. I'm running into issues with my plugin's shared object. Bro is complaining about an undefined symbol (likely due to a bug in my source). Are there plans to expand the Types section of that page? I think I'm not declaring a scriptland record type correctly. -AK On Oct 6, 2014 8:10 AM, Robin Sommer ro...@icir.orgmailto:ro...@icir.org wrote: On Sun, Oct 05, 2014 at 23:00 -0700, you wrote: I can at least now see why my plugin isn't working as excepted. Great, let me know if it's anything that the plugin documentation could make more clear. Robin -- Robin Sommer * Phone +1 (510) 722-6541tel:%2B1%20%28510%29%20722-6541 * ro...@icir.orgmailto:ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956tel:%2B1%20%28510%29%20666-2956 * www.icir.org/robinhttp://www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Oct 7, 2014, at 9:36 AM, Clark, Gilbert gc355...@ohio.edu wrote:
A mismatched function definition in .h / implementation in .cc usually causes
this issue for me. Maybe try running the mangled symbol that's reported as
missing through c++filt to see what it really is (if you haven't already) and
use nm or the like to make sure the plugin library file contains that symbol?
I ran into a similar issue recently with the approxidate plugin I've been
working on. Jon reminded me that I need to do this...
#ifdef __cplusplus
extern C {
#endif
unsigned long approxidate_relative(const char *date, const struct timeval *tv);
#ifdef __cplusplus
}
#endif
.Seth
--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
I can at least now see why my plugin isn't working as excepted. Thanks Robin. -AK On Oct 5, 2014 8:23 AM, Robin Sommer ro...@icir.org wrote: On Sun, Oct 05, 2014 at 02:30 -0700, you wrote: I'm hoping someone has a suggestion or a tip on troubleshooting the interaction between plugins and the core. If you haven't yet, try compiling Bro with --enable-debug (and then recompiling your plugin as well). Then run with -B plugins and see if debug.log says anything helpful. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Sun, Oct 05, 2014 at 23:00 -0700, you wrote: I can at least now see why my plugin isn't working as excepted. Great, let me know if it's anything that the plugin documentation could make more clear. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
Thanks Robin. Everything on the Writing Bro Plugins page is clear although the debugging section seems thin to me. Then again, I've have never compiled Bro with debugging enabled before. I'm running into issues with my plugin's shared object. Bro is complaining about an undefined symbol (likely due to a bug in my source). Are there plans to expand the Types section of that page? I think I'm not declaring a scriptland record type correctly. -AK On Oct 6, 2014 8:10 AM, Robin Sommer ro...@icir.org wrote: On Sun, Oct 05, 2014 at 23:00 -0700, you wrote: I can at least now see why my plugin isn't working as excepted. Great, let me know if it's anything that the plugin documentation could make more clear. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] Geo Location Plugin
Hello All, I recently have been looking at how Bro's plugins work. I was able to recreate the rot13 functionality from the online documentation and wanted to do something a little more challenging in hopes of learning more about plugin development. On a suggestion from Seth, I've been attempting to move the geoIP stuff, which is currently baked into the core, into a self contained plugin. I've gotten Bro and my plugin to compile together (mostly by simply moving existing code into their proper files within aux/bro-aux/plugin-support/Geo). However, I cannot seem to find the plugin listed in 'bro -NN' and I cannot use the data types or functions my bif defines. I'm hoping someone has a suggestion or a tip on troubleshooting the interaction between plugins and the core. Many thanks, -AK ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
Re: [Bro-Dev] Geo Location Plugin
On Sun, Oct 05, 2014 at 02:30 -0700, you wrote: I'm hoping someone has a suggestion or a tip on troubleshooting the interaction between plugins and the core. If you haven't yet, try compiling Bro with --enable-debug (and then recompiling your plugin as well). Then run with -B plugins and see if debug.log says anything helpful. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * ro...@icir.org ICSI/LBNL* Fax +1 (510) 666-2956 * www.icir.org/robin ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
