Re: [DISCUSS] Slack Channel Use
What we need is a slackbot where you can:
- start a discuss thread, that will get sent to email list
- listen to emails ( be on the list ) and post any discuss thread replies
not from slack TO slack
- add tagged comments to discuss thread
the list-slack singularity
On November 12, 2018 at 11:49:15, Scott C. Cote (scottcc...@gmail.com)
wrote:
I realize that I’m a “new kid” here, but I think you can have your cake and
eat it too…..
If I can create it, find it, or configure it, perhaps the really best way
is to be able to either:
1) dump public slack conversations to the developer thread - arbitrarily
2) dump public slack conversations to the user thread - IFF the
thread/conversation was tagged #user (or equivalent)
Slack is a wonderful tool for facilitating discussions - I cannot emphasize
how often spam filters and the inherent slowness email servers - have
interfered with rapid conversations. Additionally, the big “ask” of any
resolution on slack - has been “can you put this in the email thread”. Goes
without saying that the even bigger ask has been - can this be contributed
to the documentation.
I strongly recommend that you streamline the flow of information from Slack
to the list archives.
SCott
Scott C. Cote
scottcc...@gmail.com
972.900.1561
twitter: @scottccote
> On Nov 12, 2018, at 9:07 AM, Justin Leet wrote:
>
> I wanted to add back onto this thread after putting some more thought
into
> it.
>
> I like Slack for the type of small developer "what's going on here?" type
> discussions. That's the kind of thing I like being real-time ("Hey, full
> dev is acting weird", "What's the basic layout of this stuff?", "Anybody
> else seen this test failure?", etc.). I think we've been pretty good
about
> keeping our decision type dev discussions to the list (e.g. this exact
> conversation).
>
> We've been doing this more, but I would like to see more of the user and
> troubleshooting move to the list. I think we've gotten a bit better about
> it as we've settled into Slack, but having that sort of helpful stuff
> exposed and searchable for users who come in afterwards is a big selling
> point of the lists, imo.
>
> To add onto this, I'd probably like to see
>
https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> (and any other relevant links) updated to emphasize a Slack focus on
> developing Metron itself, and the user lists for configuration,
> troubleshooting, etc.
>
> Essentially, I'm proposing:
> Dev list / Jira / PRs as usual for any actual decisions + concrete
feature
> discussion/review.
> Slack for Metron development "Hey, anyone seen this or have insight or a
> starting point?" and "I'm seeing something weird in our tests" type stuff
> User list for usage and troubleshooting questions. Generally, discussions
> like this in Slack should be redirected to the user list.
>
> Is this a reasonable way separate our concerns here?
>
> On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
>> Yeah, I'm also surprised by that comment about the mailing list
activity.
>> Our dev/user list discussions are by far more active than they've ever
>> been. Just have a look at the list of DISCUSS threads that have come up
in
>> the past few months and it's clear that not only participation has
>> increased, but diversity of topic and participant.
>>
>> On Wed, Oct 24, 2018 at 8:08 AM Casey Stella wrote:
>>
>>> Not for nothing, but at least according to the last board report that I
>>> submitted, the user@ traffic is up 100% and the dev list traffic is
flat
>>> as
>>> compared to last quarter. That's not to say that we couldn't stand more
>>> discussion on the lists, but a lot of the dev discussion happens on
>> github
>>> and JIRA and I'm happy to see an uptick in user traffic.
>>>
>>> On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler
>>> wrote:
>>>
I wouldn’t be so quick to related the slack discussion with perceived
activity on the list.
That is more do to the other things that are bigger issues.
On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
>> wrote:
> I have heard recently people thought Metron is sort of dead just
>>> because
the mailing list is not so active anymore!
That is exactly my concern.
On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian
>>> wrote:
> I kind of expect to have Slack for more dev related discussions
>> rather
than
> user QA. I guess it is quite common to expect mailing list to be used
>>> for
> the purpose of knowledge sharing to make sure it will be accessible
>> by
> other users as well. Of course, it is a trade-off that most of the
>>> other
> Apache projects decided to accept the risk of keeping user related
> discussions out of Slack/IRC. However, it sometimes happens to see
>> the
> mixture of questions coming to Slack. I have heard recently people
Re: [DISCUSS] Slack Channel Use
Spot on Justin, I totally agree. My only nit is that often it's much
easier troubleshooting in Slack as opposed to the mailing lists, so I'm
game to allow some troubleshooting in Slack as long as the issue and
resolution makes it back to the lists. Given that slack message history is
being kept (although to what degree I'm not sure), we could fairly easily
link to the start of a discussion in slack in the wrap-up mailing list
email for future reference.
Jon
On Mon, Nov 12, 2018 at 2:08 PM Casey Stella wrote:
> Piling on, +1 to what Justin said.
>
> On Mon, Nov 12, 2018 at 12:42 PM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
> > I'm also +1 to Justin's points.
> >
> > On Mon, Nov 12, 2018 at 10:38 AM Nick Allen wrote:
> >
> > > +1 to all your points Justin.
> > >
> > > On Mon, Nov 12, 2018 at 10:08 AM Justin Leet
> > > wrote:
> > >
> > > > I wanted to add back onto this thread after putting some more thought
> > > into
> > > > it.
> > > >
> > > > I like Slack for the type of small developer "what's going on here?"
> > type
> > > > discussions. That's the kind of thing I like being real-time ("Hey,
> > full
> > > > dev is acting weird", "What's the basic layout of this stuff?",
> > "Anybody
> > > > else seen this test failure?", etc.). I think we've been pretty good
> > > about
> > > > keeping our decision type dev discussions to the list (e.g. this
> exact
> > > > conversation).
> > > >
> > > > We've been doing this more, but I would like to see more of the user
> > and
> > > > troubleshooting move to the list. I think we've gotten a bit better
> > > about
> > > > it as we've settled into Slack, but having that sort of helpful stuff
> > > > exposed and searchable for users who come in afterwards is a big
> > selling
> > > > point of the lists, imo.
> > > >
> > > > To add onto this, I'd probably like to see
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> > > > (and any other relevant links) updated to emphasize a Slack focus on
> > > > developing Metron itself, and the user lists for configuration,
> > > > troubleshooting, etc.
> > > >
> > > > Essentially, I'm proposing:
> > > > Dev list / Jira / PRs as usual for any actual decisions + concrete
> > > feature
> > > > discussion/review.
> > > > Slack for Metron development "Hey, anyone seen this or have insight
> or
> > a
> > > > starting point?" and "I'm seeing something weird in our tests" type
> > stuff
> > > > User list for usage and troubleshooting questions. Generally,
> > > discussions
> > > > like this in Slack should be redirected to the user list.
> > > >
> > > > Is this a reasonable way separate our concerns here?
> > > >
> > > > On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> > > > michael.miklav...@gmail.com> wrote:
> > > >
> > > > > Yeah, I'm also surprised by that comment about the mailing list
> > > activity.
> > > > > Our dev/user list discussions are by far more active than they've
> > ever
> > > > > been. Just have a look at the list of DISCUSS threads that have
> come
> > up
> > > > in
> > > > > the past few months and it's clear that not only participation has
> > > > > increased, but diversity of topic and participant.
> > > > >
> > > > > On Wed, Oct 24, 2018 at 8:08 AM Casey Stella
> > > wrote:
> > > > >
> > > > > > Not for nothing, but at least according to the last board report
> > > that I
> > > > > > submitted, the user@ traffic is up 100% and the dev list traffic
> > is
> > > > flat
> > > > > > as
> > > > > > compared to last quarter. That's not to say that we couldn't
> stand
> > > > more
> > > > > > discussion on the lists, but a lot of the dev discussion happens
> on
> > > > > github
> > > > > > and JIRA and I'm happy to see an uptick in user traffic.
> > > > > >
> > > > > > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler <
> > > ottobackwa...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I wouldn’t be so quick to related the slack discussion with
> > > perceived
> > > > > > > activity on the list.
> > > > > > > That is more do to the other things that are bigger issues.
> > > > > > >
> > > > > > >
> > > > > > > On October 24, 2018 at 07:15:30, Nick Allen (
> n...@nickallen.org)
> > > > > wrote:
> > > > > > >
> > > > > > > > I have heard recently people thought Metron is sort of dead
> > just
> > > > > > because
> > > > > > > the mailing list is not so active anymore!
> > > > > > >
> > > > > > > That is exactly my concern.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian <
> > alinazem...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > I kind of expect to have Slack for more dev related
> discussions
> > > > > rather
> > > > > > > than
> > > > > > > > user QA. I guess it is quite common to expect mailing list to
> > be
> > > > used
> > > > > > for
> > > > > > > > the purpose of knowledge sharing to make sure it will be
> > > accessible
>
Re: [DISCUSS] Slack Channel Use
Piling on, +1 to what Justin said.
On Mon, Nov 12, 2018 at 12:42 PM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:
> I'm also +1 to Justin's points.
>
> On Mon, Nov 12, 2018 at 10:38 AM Nick Allen wrote:
>
> > +1 to all your points Justin.
> >
> > On Mon, Nov 12, 2018 at 10:08 AM Justin Leet
> > wrote:
> >
> > > I wanted to add back onto this thread after putting some more thought
> > into
> > > it.
> > >
> > > I like Slack for the type of small developer "what's going on here?"
> type
> > > discussions. That's the kind of thing I like being real-time ("Hey,
> full
> > > dev is acting weird", "What's the basic layout of this stuff?",
> "Anybody
> > > else seen this test failure?", etc.). I think we've been pretty good
> > about
> > > keeping our decision type dev discussions to the list (e.g. this exact
> > > conversation).
> > >
> > > We've been doing this more, but I would like to see more of the user
> and
> > > troubleshooting move to the list. I think we've gotten a bit better
> > about
> > > it as we've settled into Slack, but having that sort of helpful stuff
> > > exposed and searchable for users who come in afterwards is a big
> selling
> > > point of the lists, imo.
> > >
> > > To add onto this, I'd probably like to see
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> > > (and any other relevant links) updated to emphasize a Slack focus on
> > > developing Metron itself, and the user lists for configuration,
> > > troubleshooting, etc.
> > >
> > > Essentially, I'm proposing:
> > > Dev list / Jira / PRs as usual for any actual decisions + concrete
> > feature
> > > discussion/review.
> > > Slack for Metron development "Hey, anyone seen this or have insight or
> a
> > > starting point?" and "I'm seeing something weird in our tests" type
> stuff
> > > User list for usage and troubleshooting questions. Generally,
> > discussions
> > > like this in Slack should be redirected to the user list.
> > >
> > > Is this a reasonable way separate our concerns here?
> > >
> > > On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> > > michael.miklav...@gmail.com> wrote:
> > >
> > > > Yeah, I'm also surprised by that comment about the mailing list
> > activity.
> > > > Our dev/user list discussions are by far more active than they've
> ever
> > > > been. Just have a look at the list of DISCUSS threads that have come
> up
> > > in
> > > > the past few months and it's clear that not only participation has
> > > > increased, but diversity of topic and participant.
> > > >
> > > > On Wed, Oct 24, 2018 at 8:08 AM Casey Stella
> > wrote:
> > > >
> > > > > Not for nothing, but at least according to the last board report
> > that I
> > > > > submitted, the user@ traffic is up 100% and the dev list traffic
> is
> > > flat
> > > > > as
> > > > > compared to last quarter. That's not to say that we couldn't stand
> > > more
> > > > > discussion on the lists, but a lot of the dev discussion happens on
> > > > github
> > > > > and JIRA and I'm happy to see an uptick in user traffic.
> > > > >
> > > > > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler <
> > ottobackwa...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I wouldn’t be so quick to related the slack discussion with
> > perceived
> > > > > > activity on the list.
> > > > > > That is more do to the other things that are bigger issues.
> > > > > >
> > > > > >
> > > > > > On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
> > > > wrote:
> > > > > >
> > > > > > > I have heard recently people thought Metron is sort of dead
> just
> > > > > because
> > > > > > the mailing list is not so active anymore!
> > > > > >
> > > > > > That is exactly my concern.
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian <
> alinazem...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > I kind of expect to have Slack for more dev related discussions
> > > > rather
> > > > > > than
> > > > > > > user QA. I guess it is quite common to expect mailing list to
> be
> > > used
> > > > > for
> > > > > > > the purpose of knowledge sharing to make sure it will be
> > accessible
> > > > by
> > > > > > > other users as well. Of course, it is a trade-off that most of
> > the
> > > > > other
> > > > > > > Apache projects decided to accept the risk of keeping user
> > related
> > > > > > > discussions out of Slack/IRC. However, it sometimes happens to
> > see
> > > > the
> > > > > > > mixture of questions coming to Slack. I have heard recently
> > people
> > > > > > thought
> > > > > > > Metron is sort of dead just because the mailing list is not so
> > > active
> > > > > > > anymore!
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Ali
> > > > > > >
> > > > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella <
> ceste...@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Agreed, the benefit of the mailing list is that it’s
> searchable
>
Re: [DISCUSS] Slack Channel Use
I'm also +1 to Justin's points.
On Mon, Nov 12, 2018 at 10:38 AM Nick Allen wrote:
> +1 to all your points Justin.
>
> On Mon, Nov 12, 2018 at 10:08 AM Justin Leet
> wrote:
>
> > I wanted to add back onto this thread after putting some more thought
> into
> > it.
> >
> > I like Slack for the type of small developer "what's going on here?" type
> > discussions. That's the kind of thing I like being real-time ("Hey, full
> > dev is acting weird", "What's the basic layout of this stuff?", "Anybody
> > else seen this test failure?", etc.). I think we've been pretty good
> about
> > keeping our decision type dev discussions to the list (e.g. this exact
> > conversation).
> >
> > We've been doing this more, but I would like to see more of the user and
> > troubleshooting move to the list. I think we've gotten a bit better
> about
> > it as we've settled into Slack, but having that sort of helpful stuff
> > exposed and searchable for users who come in afterwards is a big selling
> > point of the lists, imo.
> >
> > To add onto this, I'd probably like to see
> >
> >
> https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> > (and any other relevant links) updated to emphasize a Slack focus on
> > developing Metron itself, and the user lists for configuration,
> > troubleshooting, etc.
> >
> > Essentially, I'm proposing:
> > Dev list / Jira / PRs as usual for any actual decisions + concrete
> feature
> > discussion/review.
> > Slack for Metron development "Hey, anyone seen this or have insight or a
> > starting point?" and "I'm seeing something weird in our tests" type stuff
> > User list for usage and troubleshooting questions. Generally,
> discussions
> > like this in Slack should be redirected to the user list.
> >
> > Is this a reasonable way separate our concerns here?
> >
> > On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> > michael.miklav...@gmail.com> wrote:
> >
> > > Yeah, I'm also surprised by that comment about the mailing list
> activity.
> > > Our dev/user list discussions are by far more active than they've ever
> > > been. Just have a look at the list of DISCUSS threads that have come up
> > in
> > > the past few months and it's clear that not only participation has
> > > increased, but diversity of topic and participant.
> > >
> > > On Wed, Oct 24, 2018 at 8:08 AM Casey Stella
> wrote:
> > >
> > > > Not for nothing, but at least according to the last board report
> that I
> > > > submitted, the user@ traffic is up 100% and the dev list traffic is
> > flat
> > > > as
> > > > compared to last quarter. That's not to say that we couldn't stand
> > more
> > > > discussion on the lists, but a lot of the dev discussion happens on
> > > github
> > > > and JIRA and I'm happy to see an uptick in user traffic.
> > > >
> > > > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler <
> ottobackwa...@gmail.com>
> > > > wrote:
> > > >
> > > > > I wouldn’t be so quick to related the slack discussion with
> perceived
> > > > > activity on the list.
> > > > > That is more do to the other things that are bigger issues.
> > > > >
> > > > >
> > > > > On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
> > > wrote:
> > > > >
> > > > > > I have heard recently people thought Metron is sort of dead just
> > > > because
> > > > > the mailing list is not so active anymore!
> > > > >
> > > > > That is exactly my concern.
> > > > >
> > > > >
> > > > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian
> > > > wrote:
> > > > >
> > > > > > I kind of expect to have Slack for more dev related discussions
> > > rather
> > > > > than
> > > > > > user QA. I guess it is quite common to expect mailing list to be
> > used
> > > > for
> > > > > > the purpose of knowledge sharing to make sure it will be
> accessible
> > > by
> > > > > > other users as well. Of course, it is a trade-off that most of
> the
> > > > other
> > > > > > Apache projects decided to accept the risk of keeping user
> related
> > > > > > discussions out of Slack/IRC. However, it sometimes happens to
> see
> > > the
> > > > > > mixture of questions coming to Slack. I have heard recently
> people
> > > > > thought
> > > > > > Metron is sort of dead just because the mailing list is not so
> > active
> > > > > > anymore!
> > > > > >
> > > > > > Cheers,
> > > > > > Ali
> > > > > >
> > > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella >
> > > > wrote:
> > > > > >
> > > > > > > Agreed, the benefit of the mailing list is that it’s searchable
> > by
> > > > > > ponymail
> > > > > > > and the major search engines.
> > > > > > > On Mon, Oct 22, 2018 at 17:18 Nick Allen
> > > wrote:
> > > > > > >
> > > > > > > > I don't know that it is the same kind of searchable. Is it
> > being
> > > > > > indexed
> > > > > > > > by the major search engines? I have never used a search
> engine
> > > and
> > > > > > > > uncovered the answer to my problem in a Slack archive.
> > > > > > > >
> > > > > > > > On Mon
Re: [DISCUSS] Slack Channel Use
Your suggestion is well received. I think what we're trying to avoid is a
big dump of Slack's stream of consciousness. There is an inherent
organization and required collection of thoughts that comes with the
dev/user list discussions that doesn't occur on Slack. Maybe threads can
help that a bit, but I'm not sure it should be a full replacement.
On Mon, Nov 12, 2018 at 9:49 AM Scott C. Cote wrote:
> I realize that I’m a “new kid” here, but I think you can have your cake
> and eat it too…..
>
> If I can create it, find it, or configure it, perhaps the really best way
> is to be able to either:
>
> 1) dump public slack conversations to the developer thread - arbitrarily
> 2) dump public slack conversations to the user thread - IFF the
> thread/conversation was tagged #user (or equivalent)
>
> Slack is a wonderful tool for facilitating discussions - I cannot
> emphasize how often spam filters and the inherent slowness email servers -
> have interfered with rapid conversations. Additionally, the big “ask” of
> any resolution on slack - has been “can you put this in the email
> thread”. Goes without saying that the even bigger ask has been - can
> this be contributed to the documentation.
>
> I strongly recommend that you streamline the flow of information from
> Slack to the list archives.
>
> SCott
> Scott C. Cote
> scottcc...@gmail.com
> 972.900.1561
>
> twitter: @scottccote
>
>
>
> > On Nov 12, 2018, at 9:07 AM, Justin Leet wrote:
> >
> > I wanted to add back onto this thread after putting some more thought
> into
> > it.
> >
> > I like Slack for the type of small developer "what's going on here?" type
> > discussions. That's the kind of thing I like being real-time ("Hey, full
> > dev is acting weird", "What's the basic layout of this stuff?", "Anybody
> > else seen this test failure?", etc.). I think we've been pretty good
> about
> > keeping our decision type dev discussions to the list (e.g. this exact
> > conversation).
> >
> > We've been doing this more, but I would like to see more of the user and
> > troubleshooting move to the list. I think we've gotten a bit better
> about
> > it as we've settled into Slack, but having that sort of helpful stuff
> > exposed and searchable for users who come in afterwards is a big selling
> > point of the lists, imo.
> >
> > To add onto this, I'd probably like to see
> >
> https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> > (and any other relevant links) updated to emphasize a Slack focus on
> > developing Metron itself, and the user lists for configuration,
> > troubleshooting, etc.
> >
> > Essentially, I'm proposing:
> > Dev list / Jira / PRs as usual for any actual decisions + concrete
> feature
> > discussion/review.
> > Slack for Metron development "Hey, anyone seen this or have insight or a
> > starting point?" and "I'm seeing something weird in our tests" type stuff
> > User list for usage and troubleshooting questions. Generally,
> discussions
> > like this in Slack should be redirected to the user list.
> >
> > Is this a reasonable way separate our concerns here?
> >
> > On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> > michael.miklav...@gmail.com> wrote:
> >
> >> Yeah, I'm also surprised by that comment about the mailing list
> activity.
> >> Our dev/user list discussions are by far more active than they've ever
> >> been. Just have a look at the list of DISCUSS threads that have come up
> in
> >> the past few months and it's clear that not only participation has
> >> increased, but diversity of topic and participant.
> >>
> >> On Wed, Oct 24, 2018 at 8:08 AM Casey Stella
> wrote:
> >>
> >>> Not for nothing, but at least according to the last board report that I
> >>> submitted, the user@ traffic is up 100% and the dev list traffic is
> flat
> >>> as
> >>> compared to last quarter. That's not to say that we couldn't stand
> more
> >>> discussion on the lists, but a lot of the dev discussion happens on
> >> github
> >>> and JIRA and I'm happy to see an uptick in user traffic.
> >>>
> >>> On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler
> >>> wrote:
> >>>
> I wouldn’t be so quick to related the slack discussion with perceived
> activity on the list.
> That is more do to the other things that are bigger issues.
>
>
> On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
> >> wrote:
>
> > I have heard recently people thought Metron is sort of dead just
> >>> because
> the mailing list is not so active anymore!
>
> That is exactly my concern.
>
>
> On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian
> >>> wrote:
>
> > I kind of expect to have Slack for more dev related discussions
> >> rather
> than
> > user QA. I guess it is quite common to expect mailing list to be used
> >>> for
> > the purpose of knowledge sharing to make sure it will be accessible
> >> by
> > other
Re: [DISCUSS] Slack Channel Use
+1 to all your points Justin.
On Mon, Nov 12, 2018 at 10:08 AM Justin Leet wrote:
> I wanted to add back onto this thread after putting some more thought into
> it.
>
> I like Slack for the type of small developer "what's going on here?" type
> discussions. That's the kind of thing I like being real-time ("Hey, full
> dev is acting weird", "What's the basic layout of this stuff?", "Anybody
> else seen this test failure?", etc.). I think we've been pretty good about
> keeping our decision type dev discussions to the list (e.g. this exact
> conversation).
>
> We've been doing this more, but I would like to see more of the user and
> troubleshooting move to the list. I think we've gotten a bit better about
> it as we've settled into Slack, but having that sort of helpful stuff
> exposed and searchable for users who come in afterwards is a big selling
> point of the lists, imo.
>
> To add onto this, I'd probably like to see
>
> https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> (and any other relevant links) updated to emphasize a Slack focus on
> developing Metron itself, and the user lists for configuration,
> troubleshooting, etc.
>
> Essentially, I'm proposing:
> Dev list / Jira / PRs as usual for any actual decisions + concrete feature
> discussion/review.
> Slack for Metron development "Hey, anyone seen this or have insight or a
> starting point?" and "I'm seeing something weird in our tests" type stuff
> User list for usage and troubleshooting questions. Generally, discussions
> like this in Slack should be redirected to the user list.
>
> Is this a reasonable way separate our concerns here?
>
> On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
> > Yeah, I'm also surprised by that comment about the mailing list activity.
> > Our dev/user list discussions are by far more active than they've ever
> > been. Just have a look at the list of DISCUSS threads that have come up
> in
> > the past few months and it's clear that not only participation has
> > increased, but diversity of topic and participant.
> >
> > On Wed, Oct 24, 2018 at 8:08 AM Casey Stella wrote:
> >
> > > Not for nothing, but at least according to the last board report that I
> > > submitted, the user@ traffic is up 100% and the dev list traffic is
> flat
> > > as
> > > compared to last quarter. That's not to say that we couldn't stand
> more
> > > discussion on the lists, but a lot of the dev discussion happens on
> > github
> > > and JIRA and I'm happy to see an uptick in user traffic.
> > >
> > > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler
> > > wrote:
> > >
> > > > I wouldn’t be so quick to related the slack discussion with perceived
> > > > activity on the list.
> > > > That is more do to the other things that are bigger issues.
> > > >
> > > >
> > > > On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
> > wrote:
> > > >
> > > > > I have heard recently people thought Metron is sort of dead just
> > > because
> > > > the mailing list is not so active anymore!
> > > >
> > > > That is exactly my concern.
> > > >
> > > >
> > > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian
> > > wrote:
> > > >
> > > > > I kind of expect to have Slack for more dev related discussions
> > rather
> > > > than
> > > > > user QA. I guess it is quite common to expect mailing list to be
> used
> > > for
> > > > > the purpose of knowledge sharing to make sure it will be accessible
> > by
> > > > > other users as well. Of course, it is a trade-off that most of the
> > > other
> > > > > Apache projects decided to accept the risk of keeping user related
> > > > > discussions out of Slack/IRC. However, it sometimes happens to see
> > the
> > > > > mixture of questions coming to Slack. I have heard recently people
> > > > thought
> > > > > Metron is sort of dead just because the mailing list is not so
> active
> > > > > anymore!
> > > > >
> > > > > Cheers,
> > > > > Ali
> > > > >
> > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella
> > > wrote:
> > > > >
> > > > > > Agreed, the benefit of the mailing list is that it’s searchable
> by
> > > > > ponymail
> > > > > > and the major search engines.
> > > > > > On Mon, Oct 22, 2018 at 17:18 Nick Allen
> > wrote:
> > > > > >
> > > > > > > I don't know that it is the same kind of searchable. Is it
> being
> > > > > indexed
> > > > > > > by the major search engines? I have never used a search engine
> > and
> > > > > > > uncovered the answer to my problem in a Slack archive.
> > > > > > >
> > > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler <
> > > ottobackwa...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > According to Greg Stein, an infra admin on the NiFi slack,
> the
> > > ASF
> > > > > > slack
> > > > > > > > that metron is in IS the standard plan, not the free one and
> is
> > > > > > > searchable
> > > > > > > > past 10,000 messages.
> > > > > > > >
> >
Re: [DISCUSS] Slack Channel Use
I realize that I’m a “new kid” here, but I think you can have your cake and
eat it too…..
If I can create it, find it, or configure it, perhaps the really best way is to
be able to either:
1) dump public slack conversations to the developer thread - arbitrarily
2) dump public slack conversations to the user thread - IFF the
thread/conversation was tagged #user (or equivalent)
Slack is a wonderful tool for facilitating discussions - I cannot emphasize how
often spam filters and the inherent slowness email servers - have interfered
with rapid conversations. Additionally, the big “ask” of any resolution on
slack - has been “can you put this in the email thread”. Goes without
saying that the even bigger ask has been - can this be contributed to the
documentation.
I strongly recommend that you streamline the flow of information from Slack to
the list archives.
SCott
Scott C. Cote
scottcc...@gmail.com
972.900.1561
twitter: @scottccote
> On Nov 12, 2018, at 9:07 AM, Justin Leet wrote:
>
> I wanted to add back onto this thread after putting some more thought into
> it.
>
> I like Slack for the type of small developer "what's going on here?" type
> discussions. That's the kind of thing I like being real-time ("Hey, full
> dev is acting weird", "What's the basic layout of this stuff?", "Anybody
> else seen this test failure?", etc.). I think we've been pretty good about
> keeping our decision type dev discussions to the list (e.g. this exact
> conversation).
>
> We've been doing this more, but I would like to see more of the user and
> troubleshooting move to the list. I think we've gotten a bit better about
> it as we've settled into Slack, but having that sort of helpful stuff
> exposed and searchable for users who come in afterwards is a big selling
> point of the lists, imo.
>
> To add onto this, I'd probably like to see
> https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
> (and any other relevant links) updated to emphasize a Slack focus on
> developing Metron itself, and the user lists for configuration,
> troubleshooting, etc.
>
> Essentially, I'm proposing:
> Dev list / Jira / PRs as usual for any actual decisions + concrete feature
> discussion/review.
> Slack for Metron development "Hey, anyone seen this or have insight or a
> starting point?" and "I'm seeing something weird in our tests" type stuff
> User list for usage and troubleshooting questions. Generally, discussions
> like this in Slack should be redirected to the user list.
>
> Is this a reasonable way separate our concerns here?
>
> On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
>> Yeah, I'm also surprised by that comment about the mailing list activity.
>> Our dev/user list discussions are by far more active than they've ever
>> been. Just have a look at the list of DISCUSS threads that have come up in
>> the past few months and it's clear that not only participation has
>> increased, but diversity of topic and participant.
>>
>> On Wed, Oct 24, 2018 at 8:08 AM Casey Stella wrote:
>>
>>> Not for nothing, but at least according to the last board report that I
>>> submitted, the user@ traffic is up 100% and the dev list traffic is flat
>>> as
>>> compared to last quarter. That's not to say that we couldn't stand more
>>> discussion on the lists, but a lot of the dev discussion happens on
>> github
>>> and JIRA and I'm happy to see an uptick in user traffic.
>>>
>>> On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler
>>> wrote:
>>>
I wouldn’t be so quick to related the slack discussion with perceived
activity on the list.
That is more do to the other things that are bigger issues.
On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
>> wrote:
> I have heard recently people thought Metron is sort of dead just
>>> because
the mailing list is not so active anymore!
That is exactly my concern.
On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian
>>> wrote:
> I kind of expect to have Slack for more dev related discussions
>> rather
than
> user QA. I guess it is quite common to expect mailing list to be used
>>> for
> the purpose of knowledge sharing to make sure it will be accessible
>> by
> other users as well. Of course, it is a trade-off that most of the
>>> other
> Apache projects decided to accept the risk of keeping user related
> discussions out of Slack/IRC. However, it sometimes happens to see
>> the
> mixture of questions coming to Slack. I have heard recently people
thought
> Metron is sort of dead just because the mailing list is not so active
> anymore!
>
> Cheers,
> Ali
>
> On Tue, Oct 23, 2018 at 8:23 AM Casey Stella
>>> wrote:
>
>> Agreed, the benefit of the mailing list is that it’s searchable by
> ponymail
>> and
Re: [DISCUSS] Knox SSO feature branch review and features
What you're looking for is an OUT rewrite rule, and a filter rule on content-type. It's not spectacularly well documented, but https://knox.apache.org/books/knox-1-0-0/dev-guide.html#Rewrite+Provider and specifically https://knox.apache.org/books/knox-1-0-0/dev-guide.html#Rewrite+Steps is a starting point. There are some reasonable examples in Knox itself for the webhdfs service, which uses this mechanism: https://github.com/apache/knox/blob/master/gateway-service-webhdfs/src/main/resources/org/apache/knox/gateway/hdfs/WebHdfsDeploymentContributor/rewrite.xml Hope that helps. It's not well doc-ed sadly, and not massively flexible, but should work. I suspect from my previous experiments with this you may also need to build this file as part of the UI builds, so it is aware of the bundle names generated, because the Knox matching rules don't have proper back reference capabilities. I did a POC of this sometime back in March, that I might be able to dig out if it would help. Simon On Mon, 12 Nov 2018 at 14:59, Ryan Merriman wrote: > I'm just coming up to speed on Knox so maybe rewriting assets links are > trivial. If anyone has a good example of how to do that or can point to > some documentation, please share. > > On Mon, Nov 12, 2018 at 8:54 AM Simon Elliston Ball < > si...@simonellistonball.com> wrote: > > > Doing the Knox proxy work first certainly does make a lot of sense vs the > > SSO first approach, so I'm in favour of this. It bypasses all the > anti-CORS > > proxying stuff the other solution needed by being on the same URL space. > > > > Is there are reason we're not re-writing the asset link URLs in Knox? We > > should have a reverse content rewrite rule to avoid that problem and make > > it entirely transparent whether there is Knox or not. We shouldn't be > > changing anything about the UI services themselves. If the rewrite > service > > is complete, there is no change to base ref in the UI code, Knox would > > effectively apply it by content filtering. Note also that the gateway URL > > is configurable and likely to vary from Knox to Knox, so baking it into > the > > ng build will break non-full-dev builds. (e.g. gateway/default could well > > be gateway/xyz). > > > > I would also like to discuss removing the JDBC auth, because it's a set > of > > plaintext passwords in a mysql DB... it introduces a problematic > dependency > > (mysql) a ton of java dependencies we could cut out (JPA, eclipselink) > and > > opens up a massive security hole. I personally know of several > > organisations who are blocked from using Metron by the presence of the > JDBC > > authentication method in its current form. > > > > Simon > > > > On Mon, 12 Nov 2018 at 14:36, Ryan Merriman wrote: > > > > > Let me clarify on exposing both legacy and Knox URLs at the same time. > > The > > > base urls will look something like this: > > > > > > Legacy REST - http://node1:8082/api/v1 > > > Legacy Alerts UI - http://node1:4201:/alerts-list > > > > > > Knox REST - https://node1:8443/gateway/default/metron/api/v1 > > > Knox Alerts UI - > > > https://node1:8443/gateway/default/metron-alerts-ui/alerts-list > > > > > > If Knox were turned on and the alerts UI deployed as is, it would not > > > work. This is because static assets are referenced with > > > http://node1:4201/assets/some-asset.js which does not include the > > correct > > > context path to the alerts UI in knox. To make it work, you have to > set > > > the base ref to "/gateway/default/metron-alerts-ui" so that static > assets > > > are referenced at > > > > https://node1:8443/gateway/default/metron-alerts-ui/assets/some-asset.js > > . > > > When you do that, the legacy alerts UI will no longer work. I guess > the > > > point I'm trying to make is that we would have to switch between them > or > > > have 2 separate application running. I imagine most users only need > one > > or > > > the other running so probably not an issue. > > > > > > Jon, the primary upgrade consideration I see is with authentication. > To > > be > > > able to use Knox, you would have to upgrade to LDAP-based > authentication > > if > > > you were still using JDBC-based authentication in REST. The urls would > > > also change obviously. > > > > > > On Sun, Nov 11, 2018 at 6:38 PM zeo...@gmail.com > > wrote: > > > > > > > Phew, that was quite the thread to catch up on. > > > > > > > > I agree that this should be optional/pluggable to start, and I'm > > > interested > > > > to hear the issues as they relate to upgrading an existing cluster > > (given > > > > the suggested approach) and exposing both legacy and knox URLs at the > > > same > > > > time. > > > > > > > > Jon > > > > > > > > On Fri, Nov 9, 2018, 4:46 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> > > > > wrote: > > > > > > > > > A couple more things, and I think this goes without saying - > whatever > > > we > > > > do > > > > > with Knox should NOT > > > > > > > > > >1. Require unit and integration tests t
Re: [DISCUSS] Knox SSO feature branch review and features
Hello Ryan, I am still catching up on the architecture so let me know if I am misunderstanding anything. You could have multiple serviced deployed in Knox 1. for metron (metron/api/v1) 2. for alerts-ui (metron-alerts-ui/alerts-list) and have them run in one Knox instance and you could have one service reference from other (not recommended but possible). You can tailor rewrite rules to update the context path for the assets as well, pointed out by Simon. Knox Wiki [1] has some blogs and tutorials that you can look at. This [2] is a good tutorial on how rewriting static assets, there is also a blog [3] on basics of rewrite rules that should be a good reference. I would also be glad to look at the service definitions you have and answer any questions. [1] https://cwiki.apache.org/confluence/display/KNOX/Index [2] https://cwiki.apache.org/confluence/display/KNOX/Proxying+a+UI+using+Knox [3] https://cwiki.apache.org/confluence/display/KNOX/Proxying+a+UI+using+Knox Best, Sandeep P.S. I am a Knox committer and new to Metron. On Mon, Nov 12, 2018 at 9:59 AM Ryan Merriman wrote: > I'm just coming up to speed on Knox so maybe rewriting assets links are > trivial. If anyone has a good example of how to do that or can point to > some documentation, please share. > > On Mon, Nov 12, 2018 at 8:54 AM Simon Elliston Ball < > si...@simonellistonball.com> wrote: > > > Doing the Knox proxy work first certainly does make a lot of sense vs the > > SSO first approach, so I'm in favour of this. It bypasses all the > anti-CORS > > proxying stuff the other solution needed by being on the same URL space. > > > > Is there are reason we're not re-writing the asset link URLs in Knox? We > > should have a reverse content rewrite rule to avoid that problem and make > > it entirely transparent whether there is Knox or not. We shouldn't be > > changing anything about the UI services themselves. If the rewrite > service > > is complete, there is no change to base ref in the UI code, Knox would > > effectively apply it by content filtering. Note also that the gateway URL > > is configurable and likely to vary from Knox to Knox, so baking it into > the > > ng build will break non-full-dev builds. (e.g. gateway/default could well > > be gateway/xyz). > > > > I would also like to discuss removing the JDBC auth, because it's a set > of > > plaintext passwords in a mysql DB... it introduces a problematic > dependency > > (mysql) a ton of java dependencies we could cut out (JPA, eclipselink) > and > > opens up a massive security hole. I personally know of several > > organisations who are blocked from using Metron by the presence of the > JDBC > > authentication method in its current form. > > > > Simon > > > > On Mon, 12 Nov 2018 at 14:36, Ryan Merriman wrote: > > > > > Let me clarify on exposing both legacy and Knox URLs at the same time. > > The > > > base urls will look something like this: > > > > > > Legacy REST - http://node1:8082/api/v1 > > > Legacy Alerts UI - http://node1:4201:/alerts-list > > > > > > Knox REST - https://node1:8443/gateway/default/metron/api/v1 > > > Knox Alerts UI - > > > https://node1:8443/gateway/default/metron-alerts-ui/alerts-list > > > > > > If Knox were turned on and the alerts UI deployed as is, it would not > > > work. This is because static assets are referenced with > > > http://node1:4201/assets/some-asset.js which does not include the > > correct > > > context path to the alerts UI in knox. To make it work, you have to > set > > > the base ref to "/gateway/default/metron-alerts-ui" so that static > assets > > > are referenced at > > > > https://node1:8443/gateway/default/metron-alerts-ui/assets/some-asset.js > > . > > > When you do that, the legacy alerts UI will no longer work. I guess > the > > > point I'm trying to make is that we would have to switch between them > or > > > have 2 separate application running. I imagine most users only need > one > > or > > > the other running so probably not an issue. > > > > > > Jon, the primary upgrade consideration I see is with authentication. > To > > be > > > able to use Knox, you would have to upgrade to LDAP-based > authentication > > if > > > you were still using JDBC-based authentication in REST. The urls would > > > also change obviously. > > > > > > On Sun, Nov 11, 2018 at 6:38 PM zeo...@gmail.com > > wrote: > > > > > > > Phew, that was quite the thread to catch up on. > > > > > > > > I agree that this should be optional/pluggable to start, and I'm > > > interested > > > > to hear the issues as they relate to upgrading an existing cluster > > (given > > > > the suggested approach) and exposing both legacy and knox URLs at the > > > same > > > > time. > > > > > > > > Jon > > > > > > > > On Fri, Nov 9, 2018, 4:46 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> > > > > wrote: > > > > > > > > > A couple more things, and I think this goes without saying - > whatever > > > we > > > > do > > > > > with Knox should NOT > > > > >
Re: [DISCUSS] Slack Channel Use
I wanted to add back onto this thread after putting some more thought into
it.
I like Slack for the type of small developer "what's going on here?" type
discussions. That's the kind of thing I like being real-time ("Hey, full
dev is acting weird", "What's the basic layout of this stuff?", "Anybody
else seen this test failure?", etc.). I think we've been pretty good about
keeping our decision type dev discussions to the list (e.g. this exact
conversation).
We've been doing this more, but I would like to see more of the user and
troubleshooting move to the list. I think we've gotten a bit better about
it as we've settled into Slack, but having that sort of helpful stuff
exposed and searchable for users who come in afterwards is a big selling
point of the lists, imo.
To add onto this, I'd probably like to see
https://cwiki.apache.org/confluence/display/METRON/Community+Resources#CommunityResources-ApacheMetronCommunityResources
(and any other relevant links) updated to emphasize a Slack focus on
developing Metron itself, and the user lists for configuration,
troubleshooting, etc.
Essentially, I'm proposing:
Dev list / Jira / PRs as usual for any actual decisions + concrete feature
discussion/review.
Slack for Metron development "Hey, anyone seen this or have insight or a
starting point?" and "I'm seeing something weird in our tests" type stuff
User list for usage and troubleshooting questions. Generally, discussions
like this in Slack should be redirected to the user list.
Is this a reasonable way separate our concerns here?
On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:
> Yeah, I'm also surprised by that comment about the mailing list activity.
> Our dev/user list discussions are by far more active than they've ever
> been. Just have a look at the list of DISCUSS threads that have come up in
> the past few months and it's clear that not only participation has
> increased, but diversity of topic and participant.
>
> On Wed, Oct 24, 2018 at 8:08 AM Casey Stella wrote:
>
> > Not for nothing, but at least according to the last board report that I
> > submitted, the user@ traffic is up 100% and the dev list traffic is flat
> > as
> > compared to last quarter. That's not to say that we couldn't stand more
> > discussion on the lists, but a lot of the dev discussion happens on
> github
> > and JIRA and I'm happy to see an uptick in user traffic.
> >
> > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler
> > wrote:
> >
> > > I wouldn’t be so quick to related the slack discussion with perceived
> > > activity on the list.
> > > That is more do to the other things that are bigger issues.
> > >
> > >
> > > On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org)
> wrote:
> > >
> > > > I have heard recently people thought Metron is sort of dead just
> > because
> > > the mailing list is not so active anymore!
> > >
> > > That is exactly my concern.
> > >
> > >
> > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian
> > wrote:
> > >
> > > > I kind of expect to have Slack for more dev related discussions
> rather
> > > than
> > > > user QA. I guess it is quite common to expect mailing list to be used
> > for
> > > > the purpose of knowledge sharing to make sure it will be accessible
> by
> > > > other users as well. Of course, it is a trade-off that most of the
> > other
> > > > Apache projects decided to accept the risk of keeping user related
> > > > discussions out of Slack/IRC. However, it sometimes happens to see
> the
> > > > mixture of questions coming to Slack. I have heard recently people
> > > thought
> > > > Metron is sort of dead just because the mailing list is not so active
> > > > anymore!
> > > >
> > > > Cheers,
> > > > Ali
> > > >
> > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella
> > wrote:
> > > >
> > > > > Agreed, the benefit of the mailing list is that it’s searchable by
> > > > ponymail
> > > > > and the major search engines.
> > > > > On Mon, Oct 22, 2018 at 17:18 Nick Allen
> wrote:
> > > > >
> > > > > > I don't know that it is the same kind of searchable. Is it being
> > > > indexed
> > > > > > by the major search engines? I have never used a search engine
> and
> > > > > > uncovered the answer to my problem in a Slack archive.
> > > > > >
> > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler <
> > ottobackwa...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > According to Greg Stein, an infra admin on the NiFi slack, the
> > ASF
> > > > > slack
> > > > > > > that metron is in IS the standard plan, not the free one and is
> > > > > > searchable
> > > > > > > past 10,000 messages.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic (
> > > > > > > michael.miklav...@gmail.com) wrote:
> > > > > > >
> > > > > > > ...From an archival and broader reach point of view, I do think
> > > > there's
> > > > > > > something to be said about using the mailing list. It's also
> > easier
Re: [DISCUSS] Knox SSO feature branch review and features
I'm just coming up to speed on Knox so maybe rewriting assets links are trivial. If anyone has a good example of how to do that or can point to some documentation, please share. On Mon, Nov 12, 2018 at 8:54 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > Doing the Knox proxy work first certainly does make a lot of sense vs the > SSO first approach, so I'm in favour of this. It bypasses all the anti-CORS > proxying stuff the other solution needed by being on the same URL space. > > Is there are reason we're not re-writing the asset link URLs in Knox? We > should have a reverse content rewrite rule to avoid that problem and make > it entirely transparent whether there is Knox or not. We shouldn't be > changing anything about the UI services themselves. If the rewrite service > is complete, there is no change to base ref in the UI code, Knox would > effectively apply it by content filtering. Note also that the gateway URL > is configurable and likely to vary from Knox to Knox, so baking it into the > ng build will break non-full-dev builds. (e.g. gateway/default could well > be gateway/xyz). > > I would also like to discuss removing the JDBC auth, because it's a set of > plaintext passwords in a mysql DB... it introduces a problematic dependency > (mysql) a ton of java dependencies we could cut out (JPA, eclipselink) and > opens up a massive security hole. I personally know of several > organisations who are blocked from using Metron by the presence of the JDBC > authentication method in its current form. > > Simon > > On Mon, 12 Nov 2018 at 14:36, Ryan Merriman wrote: > > > Let me clarify on exposing both legacy and Knox URLs at the same time. > The > > base urls will look something like this: > > > > Legacy REST - http://node1:8082/api/v1 > > Legacy Alerts UI - http://node1:4201:/alerts-list > > > > Knox REST - https://node1:8443/gateway/default/metron/api/v1 > > Knox Alerts UI - > > https://node1:8443/gateway/default/metron-alerts-ui/alerts-list > > > > If Knox were turned on and the alerts UI deployed as is, it would not > > work. This is because static assets are referenced with > > http://node1:4201/assets/some-asset.js which does not include the > correct > > context path to the alerts UI in knox. To make it work, you have to set > > the base ref to "/gateway/default/metron-alerts-ui" so that static assets > > are referenced at > > https://node1:8443/gateway/default/metron-alerts-ui/assets/some-asset.js > . > > When you do that, the legacy alerts UI will no longer work. I guess the > > point I'm trying to make is that we would have to switch between them or > > have 2 separate application running. I imagine most users only need one > or > > the other running so probably not an issue. > > > > Jon, the primary upgrade consideration I see is with authentication. To > be > > able to use Knox, you would have to upgrade to LDAP-based authentication > if > > you were still using JDBC-based authentication in REST. The urls would > > also change obviously. > > > > On Sun, Nov 11, 2018 at 6:38 PM zeo...@gmail.com > wrote: > > > > > Phew, that was quite the thread to catch up on. > > > > > > I agree that this should be optional/pluggable to start, and I'm > > interested > > > to hear the issues as they relate to upgrading an existing cluster > (given > > > the suggested approach) and exposing both legacy and knox URLs at the > > same > > > time. > > > > > > Jon > > > > > > On Fri, Nov 9, 2018, 4:46 PM Michael Miklavcic < > > > michael.miklav...@gmail.com> > > > wrote: > > > > > > > A couple more things, and I think this goes without saying - whatever > > we > > > do > > > > with Knox should NOT > > > > > > > >1. Require unit and integration tests to use Knox > > > >2. Break fulldev > > > > > > > > Also, I don't know that I saw you mention this, but I'm unsure how we > > > > should leverage Knox as a core piece of the platform. i.e. should we > > make > > > > this required or optional? I'm open to hearing opinions on this, but > > I'm > > > > inclined to keep this a pluggable option. > > > > > > > > Mike > > > > > > > > > > > > On Fri, Nov 9, 2018 at 2:42 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > Thanks for the update Ryan. Per my earlier comments, I thought it > > might > > > > be > > > > > the case that we could dramatically simplify this by leveraging > > Knox's > > > > > proxy capabilities, and per your research that appears to be the > > case. > > > > This > > > > > is a dramatic simplification and improvement of this feature imo, > +1. > > > I'm > > > > > also +1 on a couple distinct steps that you've laid out: fix the UI > > > > issues > > > > > in master, then add Knox for SSO. That should help mitigate issues > > with > > > > > merge conflicts with ongoing development. > > > > > > > > > > > I think it will be a challenge exposing the UIs through both the > > Knox > > > > > url and legacy urls at the same time. > > > > > I'm not s
Re: [DISCUSS] Knox SSO feature branch review and features
Doing the Knox proxy work first certainly does make a lot of sense vs the SSO first approach, so I'm in favour of this. It bypasses all the anti-CORS proxying stuff the other solution needed by being on the same URL space. Is there are reason we're not re-writing the asset link URLs in Knox? We should have a reverse content rewrite rule to avoid that problem and make it entirely transparent whether there is Knox or not. We shouldn't be changing anything about the UI services themselves. If the rewrite service is complete, there is no change to base ref in the UI code, Knox would effectively apply it by content filtering. Note also that the gateway URL is configurable and likely to vary from Knox to Knox, so baking it into the ng build will break non-full-dev builds. (e.g. gateway/default could well be gateway/xyz). I would also like to discuss removing the JDBC auth, because it's a set of plaintext passwords in a mysql DB... it introduces a problematic dependency (mysql) a ton of java dependencies we could cut out (JPA, eclipselink) and opens up a massive security hole. I personally know of several organisations who are blocked from using Metron by the presence of the JDBC authentication method in its current form. Simon On Mon, 12 Nov 2018 at 14:36, Ryan Merriman wrote: > Let me clarify on exposing both legacy and Knox URLs at the same time. The > base urls will look something like this: > > Legacy REST - http://node1:8082/api/v1 > Legacy Alerts UI - http://node1:4201:/alerts-list > > Knox REST - https://node1:8443/gateway/default/metron/api/v1 > Knox Alerts UI - > https://node1:8443/gateway/default/metron-alerts-ui/alerts-list > > If Knox were turned on and the alerts UI deployed as is, it would not > work. This is because static assets are referenced with > http://node1:4201/assets/some-asset.js which does not include the correct > context path to the alerts UI in knox. To make it work, you have to set > the base ref to "/gateway/default/metron-alerts-ui" so that static assets > are referenced at > https://node1:8443/gateway/default/metron-alerts-ui/assets/some-asset.js. > When you do that, the legacy alerts UI will no longer work. I guess the > point I'm trying to make is that we would have to switch between them or > have 2 separate application running. I imagine most users only need one or > the other running so probably not an issue. > > Jon, the primary upgrade consideration I see is with authentication. To be > able to use Knox, you would have to upgrade to LDAP-based authentication if > you were still using JDBC-based authentication in REST. The urls would > also change obviously. > > On Sun, Nov 11, 2018 at 6:38 PM zeo...@gmail.com wrote: > > > Phew, that was quite the thread to catch up on. > > > > I agree that this should be optional/pluggable to start, and I'm > interested > > to hear the issues as they relate to upgrading an existing cluster (given > > the suggested approach) and exposing both legacy and knox URLs at the > same > > time. > > > > Jon > > > > On Fri, Nov 9, 2018, 4:46 PM Michael Miklavcic < > > michael.miklav...@gmail.com> > > wrote: > > > > > A couple more things, and I think this goes without saying - whatever > we > > do > > > with Knox should NOT > > > > > >1. Require unit and integration tests to use Knox > > >2. Break fulldev > > > > > > Also, I don't know that I saw you mention this, but I'm unsure how we > > > should leverage Knox as a core piece of the platform. i.e. should we > make > > > this required or optional? I'm open to hearing opinions on this, but > I'm > > > inclined to keep this a pluggable option. > > > > > > Mike > > > > > > > > > On Fri, Nov 9, 2018 at 2:42 PM Michael Miklavcic < > > > michael.miklav...@gmail.com> wrote: > > > > > > > Thanks for the update Ryan. Per my earlier comments, I thought it > might > > > be > > > > the case that we could dramatically simplify this by leveraging > Knox's > > > > proxy capabilities, and per your research that appears to be the > case. > > > This > > > > is a dramatic simplification and improvement of this feature imo, +1. > > I'm > > > > also +1 on a couple distinct steps that you've laid out: fix the UI > > > issues > > > > in master, then add Knox for SSO. That should help mitigate issues > with > > > > merge conflicts with ongoing development. > > > > > > > > > I think it will be a challenge exposing the UIs through both the > Knox > > > > url and legacy urls at the same time. > > > > I'm not sure I understand the issue here. Are you referring to this > > > > comment? "Added a ng build option to build the UI with base href set > to > > > > Knox base path." Isn't it just a matter of URL rewriting/forwarding? > I > > > > thought we'd be exposing the URL's directly in one context, and > through > > > > Knox in the other. Either way, it seems like we should be able to > > > provide a > > > > dynamic base path through configuration in our web applications. I'd > > > expect > > > > to modify th
Re: [DISCUSS] Knox SSO feature branch review and features
Let me clarify on exposing both legacy and Knox URLs at the same time. The base urls will look something like this: Legacy REST - http://node1:8082/api/v1 Legacy Alerts UI - http://node1:4201:/alerts-list Knox REST - https://node1:8443/gateway/default/metron/api/v1 Knox Alerts UI - https://node1:8443/gateway/default/metron-alerts-ui/alerts-list If Knox were turned on and the alerts UI deployed as is, it would not work. This is because static assets are referenced with http://node1:4201/assets/some-asset.js which does not include the correct context path to the alerts UI in knox. To make it work, you have to set the base ref to "/gateway/default/metron-alerts-ui" so that static assets are referenced at https://node1:8443/gateway/default/metron-alerts-ui/assets/some-asset.js. When you do that, the legacy alerts UI will no longer work. I guess the point I'm trying to make is that we would have to switch between them or have 2 separate application running. I imagine most users only need one or the other running so probably not an issue. Jon, the primary upgrade consideration I see is with authentication. To be able to use Knox, you would have to upgrade to LDAP-based authentication if you were still using JDBC-based authentication in REST. The urls would also change obviously. On Sun, Nov 11, 2018 at 6:38 PM zeo...@gmail.com wrote: > Phew, that was quite the thread to catch up on. > > I agree that this should be optional/pluggable to start, and I'm interested > to hear the issues as they relate to upgrading an existing cluster (given > the suggested approach) and exposing both legacy and knox URLs at the same > time. > > Jon > > On Fri, Nov 9, 2018, 4:46 PM Michael Miklavcic < > michael.miklav...@gmail.com> > wrote: > > > A couple more things, and I think this goes without saying - whatever we > do > > with Knox should NOT > > > >1. Require unit and integration tests to use Knox > >2. Break fulldev > > > > Also, I don't know that I saw you mention this, but I'm unsure how we > > should leverage Knox as a core piece of the platform. i.e. should we make > > this required or optional? I'm open to hearing opinions on this, but I'm > > inclined to keep this a pluggable option. > > > > Mike > > > > > > On Fri, Nov 9, 2018 at 2:42 PM Michael Miklavcic < > > michael.miklav...@gmail.com> wrote: > > > > > Thanks for the update Ryan. Per my earlier comments, I thought it might > > be > > > the case that we could dramatically simplify this by leveraging Knox's > > > proxy capabilities, and per your research that appears to be the case. > > This > > > is a dramatic simplification and improvement of this feature imo, +1. > I'm > > > also +1 on a couple distinct steps that you've laid out: fix the UI > > issues > > > in master, then add Knox for SSO. That should help mitigate issues with > > > merge conflicts with ongoing development. > > > > > > > I think it will be a challenge exposing the UIs through both the Knox > > > url and legacy urls at the same time. > > > I'm not sure I understand the issue here. Are you referring to this > > > comment? "Added a ng build option to build the UI with base href set to > > > Knox base path." Isn't it just a matter of URL rewriting/forwarding? I > > > thought we'd be exposing the URL's directly in one context, and through > > > Knox in the other. Either way, it seems like we should be able to > > provide a > > > dynamic base path through configuration in our web applications. I'd > > expect > > > to modify that property based on whether Knox is configured or not. > > > > > > > I'm also not clear on how one would use Knox with REST set to legacy > > > JDBC-based authentication. As far as I know Knox does not support JDBC > so > > > there would be a mismatch between Knox and REST. > > > I'm OK with not having Knox work with JDBC. That's a feature of Knox > and > > > probably not something we care much about. > > > > > > >We could initially make Knox an optional feature that requires setup > > with > > > the help of some documentation (like Kerberos) while keeping the system > > the > > > way it is now by default. > > > Sounds good to me. > > > > > > > I imagine we'll deprecate JDBC-based authentication at some point so > > > that may be a good time to switch. > > > I would like to announce deprecation in our next release and move to > > > remove it in a following release. > > > > > > Thanks for taking this on and great job laying things out. > > > > > > Thanks, > > > Mike > > > > > > On Fri, Nov 9, 2018 at 2:09 PM Ryan Merriman > > wrote: > > > > > >> I have spent some time recently reviewing this discussion and the > > feature > > >> branch that Simon put out. I think this is an important feature and > > want > > >> to move it forward. I started another discussion on adding Knox to > our > > >> stack but this discussion has a lot of good context so I will continue > > it > > >> here. > > >> > > >> I think the main point of contention was that this feature branch
