(1) I am not sure if licensing is a problem here. (2) I am OK with whatever we need to get this effort done and under ASF.
On Wed, Apr 5, 2017 at 1:12 PM, zeo...@gmail.com <zeo...@gmail.com> wrote: > I'm working on this > <https://github.com/JonZeolla/incubator-metron/tree/METRON-348> in > preparation for the new repo and migration to a package. It looks like in > bro-plugins COPYING > <https://github.com/bro/bro-plugins/blob/master/kafka/COPYING> attributes > to Nick, but in our version COPYING > <https://github.com/apache/incubator-metron/tree/master/ > metron-sensors/bro-plugin-kafka/COPYING> > points to the Apache License. Same with MAINTAINER (this > <https://github.com/apache/incubator-metron/blob/master/ > metron-sensors/bro-plugin-kafka/MAINTAINER> > vs this <https://github.com/bro/bro-plugins/blob/master/kafka/MAINTAINER > >). > I assume when we package this up and host it in Apache we need to give it > the Apache license, and point to Metron for MAINTAINER. My questions are: > > 1. Is there any legal/licensing concern here? I am taking changes from > the bro-plugins version and pulling it into the Apache-hosted code. IANAL > 2. Nick - are you OK with these changes? > > Jon > > On Mon, Apr 3, 2017 at 3:50 PM zeo...@gmail.com <zeo...@gmail.com> wrote: > > > Can someone on the PMC submit a ticket to INFRA? It looks like > > <https://www.apache.org/dev/infra-contact> committers aren't supposed > to. > > > > Jon > > > > On Fri, Mar 31, 2017 at 4:23 PM zeo...@gmail.com <zeo...@gmail.com> > wrote: > > > > I would be happy to try it again but I attempted to do that before with > > bro packages and it failed to be able to handle it. I also tried using > > branches of a repo with bro but that similarly failed (and was a pretty > bad > > idea to start with). > > > > Jon > > > > On Fri, Mar 31, 2017, 3:24 PM Matt Foley <ma...@apache.org> wrote: > > > > We should be able to request just one alternate repo from INFRA, and put > a > > top hierarchical level in it that doesn’t include a maven pom. As far as > > maven and clients are concerned, it > > > > just increases by 1 the path length to the root of the repo. > > > > On 3/31/17, 10:30 AM, "zeo...@gmail.com" <zeo...@gmail.com> wrote: > > > > Once we agree on a repo location to host this, I would be happy to > put > > together the package and update our environments to use bro-pkg to > > install > > the plugin. I have created METRON-813 > > <https://issues.apache.org/jira/browse/METRON-813> to track this and > > changed METRON-348 <https://issues.apache.org/jira/browse/METRON-348 > > > > to be > > a sub-task. > > > > Otto - the bro packages model doesn't allow colocation with anything > > else. > > That said, if we have two similar situations, and given the INFRA > > example > > <https://issues.apache.org/jira/browse/INFRA-7060> Casey linked to > > before > > was requesting 9 repos, perhaps we just request two repos. Would > > someone > > else mind putting that request in? > > > > Jon > > > > On Fri, Mar 31, 2017 at 12:49 PM Otto Fowler < > ottobackwa...@gmail.com> > > wrote: > > > > Could we create a separate repo for more than on thing? like put … > um > > let’s say > > a maven plugin and the bro plugin? > > > > > > > > On March 31, 2017 at 12:30:25, Nick Allen (n...@nickallen.org) > wrote: > > > > I agree with everything that I've read. > > > > One of the guys from Bro had contacted me a while back, letting me > know > > that the packaging mechanism in Bro was ready for public > consumption. I > > just have not had cycles to do anything with it yet. They are not > > wanting > > to host any of the plugins. > > > > I thought the package mechanism requires that a package live within > > its own > > repo (which Casey confirmed). This put me in a bind on how to tackle > > this. I don't want to personally host the plugin in my own Github > > repo. I > > would prefer that we host it in a community repo; either Bro or > Metron. > > Since Bro is moving away from hosting their own plugins, that leaves > > Metron. > > > > It would be great if we could create a separate repo for the plugin. > > That > > solves the challenge of using the packaging mechanism. > > > > We do need to reconcile what is in bro/bro-plugins and what is in > > Metron. > > There are some enhancements that I and others have made that never > > made it > > back into Metron. They never made it back, because the original plan > > was > > just to switch to using the plugin from bro/bro-plugins before the > > idea of > > a packaging mechanism hit Bro. Reconciling should be fairly easy to > > see by > > just doing a diff. > > > > It would be great if others want to take on any of that work. I would > > be > > glad to offer any support that you need. Thanks, Jon! > > > > > > > > > > > > On Thu, Mar 30, 2017 at 11:20 PM, zeo...@gmail.com <zeo...@gmail.com > > > > wrote: > > > > > Ok, great. > > > > > > I agree, I definitely want to hear from Nick on the topic. My team > is > > > currently looking into enhancing the plugin as well to potentially > > allow > > > sending to multiple clusters, investigating some issues we see when > > our > > bro > > > cluster is under load, turn it into a package, etc. > > > > > > The work you just did was on our to do list as well so I'm very > > excited > > to > > > see it come through. > > > > > > Jon > > > > > > On Thu, Mar 30, 2017, 11:16 PM Casey Stella <ceste...@gmail.com> > > wrote: > > > > > > I *think* it's possible. People do ask for mirrors of directories > > from > > > time to time (see https://issues.apache.org/jira/browse/INFRA-7060 > ). > > If > > > we > > > think this is a good idea, we can pose it to INFRA as a request. > I'd > > love > > > to see us be able to use the bro packaging infrastructure and get > > more > > > visibility for the plugin. > > > > > > I'd be particularly interested in Nick's opinion on this, though. > > > > > > On Thu, Mar 30, 2017 at 11:12 PM, zeo...@gmail.com < > zeo...@gmail.com > > > > > > wrote: > > > > > > > You can version packages - > > > > http://bro-package-manager.readthedocs.io/en/stable/ > > > package.html#package- > > > > versioning > > > > > > > > I agree that having a separate repo provided by Apache would be > > optimal, > > > I > > > > just don't know the process for that or if it was even reasonable > > to > > > > suggest. > > > > > > > > Jon > > > > > > > > On Thu, Mar 30, 2017, 11:01 PM Casey Stella <ceste...@gmail.com> > > wrote: > > > > > > > > > Looking at the bro packages, it appears that bro is expecting > > things > > to > > > > be > > > > > its own git repository. I wonder if we could either request > INFRA > > > > provide > > > > > another repo for the bro-kafka plugin and integrate it into > > metron as > > a > > > > git > > > > > submodule *or* if we could request INFRA to create a github > > mirror of > > > the > > > > > metron-sensors/bro-kafka-plugin directory. I'm not sure how > > viable > > > > either > > > > > of those options are, frankly. > > > > > > > > > > One thing that I didn't see is how do you specify a particular > > release > > > of > > > > > the plugin that you want to install? For us, we'd want to > > release the > > > > > plugin along with the product. I didn't quite see how you'd > push > > > > releases > > > > > for bro plugins. > > > > > > > > > > On Thu, Mar 30, 2017 at 10:49 PM, Casey Stella < > > ceste...@gmail.com> > > > > wrote: > > > > > > > > > > > So, I do agree with the concern. Is there a way to host the > > package > > > > > > within Metron? I definitely would like to see the > > modifications at > > > > > > https://github.com/bro/bro-plugins/commit/b9f1f35415cb0db > > > > > > 065348da0a5043a8353b4a0a8 brought back into Metron and I'd > > love for > > > us > > > > to > > > > > > host the plugin. > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > On Thu, Mar 30, 2017 at 9:09 PM, zeo...@gmail.com < > > zeo...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > >> Today I was taking a look at METRON-812 > > > > > >> <https://issues.apache.org/jira/browse/METRON-812>, which > > made me > > > > > recall > > > > > >> some conversations from a while back regarding where the bro > > kafka > > > > > plugin > > > > > >> should ultimately live, and how to update it. > > > > > >> > > > > > >> Back in METRON-348 <https://issues.apache.org/ > > > jira/browse/METRON-348> > > > > I > > > > > >> brought up the fact that some important changes > > > > > >> <https://github.com/bro/bro-plugins/commit/ > b9f1f35415cb0db06 > > > > > >> 5348da0a5043a8353b4a0a8> > > > > > >> were made to the externally hosted version of the kafka > > plugin, > > and > > > > were > > > > > >> never introduced to Metron's hosted version (i.e. the one we > > use > > > > > >> <https://github.com/apache/incubator-metron/blob/master/ > metr > > > > > >> on-deployment/roles/bro/tasks/bro-plugin-kafka.yml> > > > > > >> in vagrant when bro is installed). The conversation went > down > > the > > > > route > > > > > >> of > > > > > >> discussing whether or not the bro kafka plugin code should > > continue > > > to > > > > > >> live > > > > > >> in Metron in the first place. Now, with METRON-812, I see us > > > further > > > > > >> muddying the waters of where to go for the right plugin, as > > our > > > > version > > > > > is > > > > > >> still missing the public changes but adds some very > important > > new > > > > > >> functionality. > > > > > >> > > > > > >> I'd like to bring up the idea of using bro's packages > > > > > >> <https://github.com/bro/packages> framework, released in > > late 2016 > > > > > >> < > > http://blog.bro.org/2016/10/introducing-bro-package-manager.html> > > > > > >> (additional > > > > > >> documentation here < > > > > > http://bro-package-manager.readthedocs.io/en/stable/ > > > > > >> >), > > > > > >> as a potential place for this to be hosted/referenced. This > > is a > > > > simple > > > > > >> and supported method (funded by Mozilla > > > > > >> <https://blog.mozilla.org/blog/2015/12/10/mozilla-open- > sourc > > > > > >> e-support-first-awards-made/>) > > > > > >> to install and uninstall bro scripts, plugins, etc., and it > > also > > > > allows > > > > > us > > > > > >> to continue to have enough control over updates to the > plugin > > so > > > that > > > > it > > > > > >> will not slow down Metron development by having it as a > > dependency > > > > > >> (resolving both of Casey's concerns noted here > > > > > >> <https://issues.apache.org/jira/browse/METRON-348? > focusedCom > > > > > >> mentId=15391865&page=com.atlassian.jira.plugin.system. > > > > > >> issuetabpanels:comment-tabpanel#comment-15391865>, > > > > > >> and I think this solution is supported by Nick's comments > here > > > > > >> <https://issues.apache.org/jira/browse/METRON-348? > focusedCom > > > > > >> mentId=15391872&page=com.atlassian.jira.plugin.system. > > > > > >> issuetabpanels:comment-tabpanel#comment-15391872> > > > > > >> as > > > > > >> well). > > > > > >> > > > > > >> The only thing I'm not sure about is where to host the > plugin > > itself > > > - > > > > > my > > > > > >> first thought would be Nick's github < > > https://github.com/nickwallen > > > >, > > > > > as > > > > > >> he > > > > > >> really kicked off this effort, but maybe we can think of > > something > > > > > better. > > > > > >> > > > > > >> Is this approach of interest to anybody? It is extremely > > simple to > > > > put > > > > > >> together - I was able to throw one together > > > > > >> <https://github.com/bro/packages/blob/master/ > > > jonzeolla/bro-pkg.index> > > > > > and > > > > > >> get it working with a fresh bro 2.5 install when attending > the > > > brocon > > > > > talk > > > > > >> <https://www.bro.org/brocon2016/brocon2016_abstracts.html# > > > > > >> bro-packagemanager> > > > > > >> (recording <https://www.youtube.com/watch?v=9RFfPJeGkcE>, > > slides > > > > > >> <https://www.bro.org/brocon2016/slides/hall_bpm.pdf>) that > > > introduced > > > > > >> this > > > > > >> to me in the first place. > > > > > >> > > > > > >> Jon > > > > > >> -- > > > > > >> > > > > > >> Jon > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Jon > > > > > > > > > > -- > > > > > > Jon > > > > > > > -- > > > > Jon > > > > > > > > -- > > > > Jon > > > > -- > > > > Jon > > > -- > > Jon >
