Re: [Bro-Dev] Moving to GitHub?

[Robin Sommer]

> * Someone is likely to report the same problem again
> * There's clear directions on how to reproduce an undesired behavior
> * There been a proposed plan of action recently
> 
> And many tickets can be ruled out:
> 
> * Vague feature requests
> * Not enough details  / difficult to reproduce
> * Exceptionally old proposals / plans

Yeah, I'm on board with these. I'd probably interpret them more
conservatively than you towards closing more tickets, but that's fine.
As you have volunteered to take this one, I'd say you get to make the
call: just go through and port over what you think makes sense along
those lines. As one additional piece, let's also think about some tags
to use for classifying tickets, including one for what's good tasks
for newcomers who want to get into the code.

(In principle I also like Alan's suggestion of moving everything over
and then just close them out so that the history remains. But I'm
afraid that couldn't be automated easily and would then just be too
much work.)

> starting with a clean slate on GitHub now only means it's likely to
> eventually end up in the same situation as JIRA later.  What then?
> Move to another tracker again?

Doesn't need to be as drastic: as some people here can confirm, I
have no problem doing extensive sweeps if things get too overwhelming. :-)

But yes, point taken, my hope is that we can stay on top of things on
the new tracker and make an effort to get stuff addressed and
resolved. We'll see. :)

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Jon Siwek]

On 5/18/18 11:11 AM, Robin Sommer wrote:

> What I was envisioning is more or less a clean slate: we'd migrate
> over a few tickets, but essentially we'd start with an empty list. I
> realize that sounds pretty harsh. However, I hardly ever see any
> activity on older tickets in JIRA, and I generally believe that the
> less open tickets a tracker has, the easier it is for people to
> understand what's actually relevant and worth spending cycles on.

I see those as independent issues:

(1) There's many open tickets: you solve that by actually addressing the 
tickets

(2) People don't know what tickets are relevant: you solve that by 
organizing (and maintaining) them according to relevance

My take is that there's been a lack of effort in (2) and if that's not 
solved, starting with a clean slate on GitHub now only means it's likely 
to eventually end up in the same situation as JIRA later.  What then? 
Move to another tracker again?

> Tagging tickets may help, but in the end if everybody just filters 95%
> out all the time anyways, I'm not sure what the value is.

The value is actually having a central place for all tickets and knowing 
there won't be ongoing hassle with keeping JIRA updated.

Or in reverse, I'm not sure what the value of keeping JIRA open at all 
is -- we either have to acknowledge its potential to go out of sync with 
GH in terms of duplicate reports, which makes it more of a hindrance 
IMO, or we still have to put effort to maintain it in addition to GH.

> That said, I'm open to a real porting effort if people do believe it's
> helpful to get all the JIRA tickets into GitHub. What do others think?

Pedantically, I'm not saying "port all JIRA tickets", I still want just 
the "valid" ones whatever we decide that to mean.  So more 
clarifications on potential porting criteria:

* Someone is likely to report the same problem again
* There's clear directions on how to reproduce an undesired behavior
* There been a proposed plan of action recently

And many tickets can be ruled out:

* Vague feature requests
* Not enough details  / difficult to reproduce
* Exceptionally old proposals / plans

My plea is: extend the porting criteria beyond "important" and "recent", 
perform a review of all existing tickets, and, if JIRA is going to stick 
around as a read-only archive, leave it in a coherent final state.

- Jon
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Alan Commike]

On Fri, May 18, 2018 at 9:12 AM Robin Sommer  wrote:

>
>
> That said, I'm open to a real porting effort if people do believe it's
> helpful to get all the JIRA tickets into GitHub. What do others think?
>

Having the historical tickets available are useful for searching to see if
someone else had a similar issue and if there's a possible work-around.

How about a solution somewhere in the middle - push all the tickets over
but mark the older/non-critical as 'won't fix'. They'll come up in searches
and can more easily be brought back alive if needed.

   ...alan


>
> Robin
>
> --
> Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
> ___
> bro-dev mailing list
> bro-dev@bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Johanna Amann]

I am also more in favor of starting clean and manually letting people 
move tickets that they think are important over.

But - currently there is a lot in the tracker that are nice to have or 
potential problems that I do not ever see getting addressed.

Johanna

On 18 May 2018, at 9:17, Slagell, Adam J wrote:

> On May 18, 2018, at 11:11 AM, Robin Sommer 
> > wrote:
>
> What I was envisioning is more or less a clean slate: we'd migrate
> over a few tickets, but essentially we'd start with an empty list. I
> realize that sounds pretty harsh. However, I hardly ever see any
> activity on older tickets in JIRA, and I generally believe that the
> less open tickets a tracker has, the easier it is for people to
> understand what's actually relevant and worth spending cycles on.
> Tagging tickets may help, but in the end if everybody just filters 95%
> out all the time anyways, I'm not sure what the value is.
>
> That said, I'm open to a real porting effort if people do believe it's
> helpful to get all the JIRA tickets into GitHub. What do others think?
>
> Start clean for BIT. Unless it is marked critical, I don’t think it 
> needs to go over. If people have tickets of their own they want to 
> move over, it should not be too hard to manually recreate a couple.
>
> And after BroCon I think we kill the Jira and wiki completely. We 
> should be done with the project management and event tickets by then.
>
> --
>
> Adam J. Slagell
> Director, Cybersecurity & Networking Division
> Chief Information Security Officer
> National Center for Supercomputing Applications
> University of Illinois at Urbana-Champaign
> www.slagell.info
>
> "Under the Illinois Freedom of Information Act (FOIA), any written 
> communication to or from University employees regarding University 
> business is a public record and may be subject to public disclosure."
>
>
>
>
>
>
>
>
> ___
> bro-dev mailing list
> bro-dev@bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Slagell, Adam J]

On May 18, 2018, at 11:11 AM, Robin Sommer 
> wrote:

What I was envisioning is more or less a clean slate: we'd migrate
over a few tickets, but essentially we'd start with an empty list. I
realize that sounds pretty harsh. However, I hardly ever see any
activity on older tickets in JIRA, and I generally believe that the
less open tickets a tracker has, the easier it is for people to
understand what's actually relevant and worth spending cycles on.
Tagging tickets may help, but in the end if everybody just filters 95%
out all the time anyways, I'm not sure what the value is.

That said, I'm open to a real porting effort if people do believe it's
helpful to get all the JIRA tickets into GitHub. What do others think?

Start clean for BIT. Unless it is marked critical, I don’t think it needs to go 
over. If people have tickets of their own they want to move over, it should not 
be too hard to manually recreate a couple.

And after BroCon I think we kill the Jira and wiki completely. We should be 
done with the project management and event tickets by then.

--

Adam J. Slagell
Director, Cybersecurity & Networking Division
Chief Information Security Officer
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written 
communication to or from University employees regarding University business is 
a public record and may be subject to public disclosure."








___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Robin Sommer]

On Fri, May 18, 2018 at 08:27 -0500, you wrote:

> Doing a half-hearted effort to migrate tickets from JIRA undermines the goal
> of having an authoritative/central location for all code + tickets.  Can we
> instead try to deal with it once and for all?

What I was envisioning is more or less a clean slate: we'd migrate
over a few tickets, but essentially we'd start with an empty list. I
realize that sounds pretty harsh. However, I hardly ever see any
activity on older tickets in JIRA, and I generally believe that the
less open tickets a tracker has, the easier it is for people to
understand what's actually relevant and worth spending cycles on.
Tagging tickets may help, but in the end if everybody just filters 95%
out all the time anyways, I'm not sure what the value is.

That said, I'm open to a real porting effort if people do believe it's
helpful to get all the JIRA tickets into GitHub. What do others think?

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Jon Siwek]

On 5/17/18 6:27 PM, Robin Sommer wrote:

> That may be a bit too broad though. How about "still valid and either
> (1) quite important or (2) something we expect will be addresses
> reasonably soon"? We have many old tickets that are technically still
> valid but unlikely to see any work anytime soon (otherwise they would
> have been addressed already), and I'm worried that they would just add
> noise without value.

That sounds like a general concern about project/ticket organization and 
management.  What if valid-but-old tickets were moved into GitHub with a 
simple "backlog" tag that you can filter out?  Or we could take the 
opportunity to be better about sorting on other categories as well 
(bug/feature/etc).

There's also the possibility to start a project board page [1] to aid in 
visual organization of tasks/issues and further reduce perceived noise. 
See [2] for an example of what one of those pages could look like.

> The old tickets won't go away, the JIRA will
> remain. If something becomes relevant/active, we can always bring it
> over at that time.

Keeping the JIRA around as a backlog like that decreases 
focus/visibility -- if we want to be optimistic about community 
contributions and bug fixes, we'd have to keep calling attention to 
search in two locations, GH and JIRA, instead of just one.  We'd also 
need to stay on top of maintaining JIRA to be relatively in sync with 
anything that gets resolved in GH else keeping JIRA around may become 
more liability than useful as it gets harder and harder to tell if 
something there has actually been addressed.

Doing a half-hearted effort to migrate tickets from JIRA undermines the 
goal of having an authoritative/central location for all code + tickets. 
  Can we instead try to deal with it once and for all?

- Jon

[1] https://github.com/orgs/bro/projects
[2] https://github.com/git-lfs/git-lfs/projects/3
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Robin Sommer]

On Thu, May 17, 2018 at 11:21 -0500, you wrote:

> * For porting over JIRA tickets to GitHub, "most recent" doesn't seem like a
> good metric to use.

Agree. :)

> they may as well just port all the older ones that are still valid
> over to GitHub.

That may be a bit too broad though. How about "still valid and either
(1) quite important or (2) something we expect will be addresses
reasonably soon"? We have many old tickets that are technically still
valid but unlikely to see any work anytime soon (otherwise they would
have been addressed already), and I'm worried that they would just add
noise without value. The old tickets won't go away, the JIRA will
remain. If something becomes relevant/active, we can always bring it
over at that time.

> I find myself in that situation quite often, actually, so
> transitioning to GitHub PRs, I wonder if we'd want a PR to be created
> against each individual repo?

Good point. Creating just one root PR that mentions the others sounds
good to me for such cases. 

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Seth Hall]

On 15 May 2018, at 20:19, Robin Sommer wrote:

> This has been coming up in various contexts & subgroups of people, and
> I wanted to send it out as a proposal to gather some broader feedback:
> Do we want to move Bro's git repositories and tickets to GitHub?

I like the idea.  It'll be nice to have one process for handling 
everything instead of the multiple avenues for tickets and merge 
requests that we have today.

It seems that we should even be able to solve the issue with the 
immediate diff emails by using webhooks with an AWS lambda function.  
I'm not sure if we'd be able to do all of the git work from that though.

   .Seth

--
Seth Hall * Corelight, Inc * www.corelight.com
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Jon Siwek]

On 5/15/18 7:19 PM, Robin Sommer wrote:
> What do people think? Any support, or concerns?

Yeah, generally in favor with some comments:

* For porting over JIRA tickets to GitHub, "most recent" doesn't seem 
like a good metric to use.  e.g. BIT-1829 (pcap triggering assertion in 
binpac) seems kind of important and not something I'd want to have lost 
in the move, although it had no activity in almost a year.  So, I think 
it's worth it to be more comprehensive here, and as long as someone is 
going through to review all the tickets, they may as well just port all 
the older ones that are still valid over to GitHub.  (Yeah, I guess I'm 
volunteering).

* One thing I did like about using JIRA for merge requests is that I 
could make a single ticket and just say I have a given branch name in a 
bunch of repos that are ready for review/merge.  I find myself in that 
situation quite often, actually, so transitioning to GitHub PRs, I 
wonder if we'd want a PR to be created against each individual repo? 
Seems a bit much in terms of overhead.  Alternatively, could still 
create a GH issue and just say "please review branch foo in repos X, Y, 
Z and merge them".  Or else create a single PR in the "root" repo and 
mention in the PR that the same branch in child submodules also exists 
and needs merging.  That may play better with Travis CI integration 
even, although maybe not in the case where you need to change things in 
the "external testing" repos, which are not connected to bro via a 
submodule.

- Jon
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Robin Sommer]

On Wed, May 16, 2018 at 15:23 +, you wrote:

> I too would miss the commit / change notifications, however, I think
> that this can be set up in GitHub in some way.

We can still get the same email notifications as today (which have a
bit more information that GitHub's standard ones), they will just come
with a little bit of a delay (within 10-15 minutes should be
reasonable). And if we want we can trigger that through webbhooks,
too, for immediate notification, would just need a bit of work to get
it set up.

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Hosom, Stephen M]

Well... in looking at how I set this up internally, I now realize that this 
feature is being EOL and will no longer work as of January 31, 2019. I guess it 
would have to be webhooks.

On 5/16/18, 11:30 AM, "bro-dev-boun...@bro.org on behalf of Hosom, Stephen M" 
 wrote:

I too would miss the commit / change notifications, however, I think that 
this can be set up in GitHub in some way. We use Slack / Email internally for 
commit notifications. For Slack, we use webhooks to send notifications. 

Email commit notifications are actually a builtin integration in GitHub... 
You can set them up under Settings -> Integrations & Services -> Add service -> 
Email. 


On 5/15/18, 8:41 PM, "bro-dev-boun...@bro.org on behalf of Johanna Amann" 
 wrote:

Message received from outside the Battelle network. Carefully examine 
it before you open any links or attachments.

> What do people think? Any support, or concerns?

I am in favor. The only thing I would miss are the immediate change 
notifications by email - I really like those...

Johanna
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev



___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev



___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Hosom, Stephen M]

I too would miss the commit / change notifications, however, I think that this 
can be set up in GitHub in some way. We use Slack / Email internally for commit 
notifications. For Slack, we use webhooks to send notifications. 

Email commit notifications are actually a builtin integration in GitHub... You 
can set them up under Settings -> Integrations & Services -> Add service -> 
Email. 


On 5/15/18, 8:41 PM, "bro-dev-boun...@bro.org on behalf of Johanna Amann" 
 wrote:

Message received from outside the Battelle network. Carefully examine it 
before you open any links or attachments.

> What do people think? Any support, or concerns?

I am in favor. The only thing I would miss are the immediate change 
notifications by email - I really like those...

Johanna
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev



___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Michał Purzyński]

Big thumbs up. 

On May 15, 2018, at 5:34 PM, Johanna Amann  wrote:

>> What do people think? Any support, or concerns?
> 
> I am in favor. The only thing I would miss are the immediate change 
> notifications by email - I really like those...
> 
> Johanna
> ___
> bro-dev mailing list
> bro-dev@bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] Moving to GitHub?

[Slagell, Adam J]

I am in favor. I would like to still maintain the the Jira and wiki for a 
couple months until we finish up some work. Really just the BPM tickets. 

> On May 15, 2018, at 7:19 PM, Robin Sommer  wrote:
> 
> This has been coming up in various contexts & subgroups of people, and
> I wanted to send it out as a proposal to gather some broader feedback:
> Do we want to move Bro's git repositories and tickets to GitHub?
> 
> Currently we're hosting our Git repositories on our own server at
> git.bro.org; from there, we mirror them to GitHub. For issue tracking,
> we use JIRA at tracker.bro.org. Much of all this is a legacy setup in
> some sense. I believe it would simplify life for both users &
> developers if we moved to GitHub as the authoratative place for both
> code and tickets.
> 
> More specifically, I propose that we:
> 
> 1. Turn the current git repository mirrors on github.com/bro into
>authoritative source for all Bro code. Then we discontinue
>git.bro.org. We can set up up some notification system there
>instead that points people still using the old URLs to GitHub.
> 
> 2. Switch to using GitHub as our primary issue tracker. Giving
>the large amount of old tickets in JIRA, I think we should
>just start from scratch there, porting over just the most
>recent pending tickets. We'd keep the JIRA running in
>read-only mode so that we don't loose the history and can
>always refer back to old tickets.
> 
> 3. Switch to a mostly PR-based development model (i.e., no more
>merge requests tracked separately through tickets), and also
>use GitHub for code review & feedback.
> 
> 4. Make sure it all integrates nicely with Travis CI (which has
>already been set up recently).
> 
> About the only downside I see is that emailing out our standard commit
> notifications won't be quite as smooth anymore: we'd still get them,
> but with a bit of delay as some system would need to be polling for
> changes, rather than being triggered immediately. Not much of a
> problem I think (and with some additional work, we could make them
> push-triggered, too; but probably not worth it).
> 
> What do people think? Any support, or concerns?
> 
> Robin
> 
> -- 
> Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
> ___
> bro-dev mailing list
> bro-dev@bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev