I just put up a PR that adds Metron as a Knox service here:
https://github.com/apache/metron/pull/1275. This should give everyone a
good idea of what is involved. I added a section on outstanding items that
highlights some of the things we have been discussion here.
On Fri, Nov 16, 2018 at
I would also add that defaulting to Knox being on simplifies things at a
technical level.
On Fri, Nov 16, 2018 at 10:52 AM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:
> That's fantastic, thanks for that detail.
>
> Also, I'm in agreement with the recent comments from Otto and Simon.
That's fantastic, thanks for that detail.
Also, I'm in agreement with the recent comments from Otto and Simon.
On Fri, Nov 16, 2018 at 9:49 AM Ryan Merriman wrote:
> I was still able to spin up the UI locally and debug in my testing. I am
> in complete agreement, we need to ensure the
I was still able to spin up the UI locally and debug in my testing. I am
in complete agreement, we need to ensure the developer experience doesn't
change.
On Fri, Nov 16, 2018 at 10:47 AM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:
> Ryan, what's remote debugging look like for UI
Ryan, what's remote debugging look like for UI testing with Knox enabled?
Anything we lose from a dev testability standpoint? The discussion of
defaults sounds reasonable to me, and I'd like to understand any other
tradeoffs there may be for non-prod deployments like full dev.
On Fri, Nov 16,
It is included, yes, but is not started out of the box by default, we would
also probably tweak the blueprint to change its bootstrap ldif file a bit
to have more sensible user names for our defaults, but that's pretty simple
load. A bit of default blueprint config is what we need there, not
That does sound good Simon, I think I miss understood that the default LDAP
was standard with KNOX/ambari and not something we would be doing ourselves.
On November 16, 2018 at 10:54:48, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
I think there is a lot to be said for defaulting
I think there is a lot to be said for defaulting to Knox on... we also that
way get some 'secure by default' - at least ssl by default. The do-nothing
I think you're proposing would be around the authentication, right? Knox
does ship with a demo LDAP server we could have some defaults (kinda like
Those are all valid points. I think it is ( was ) worth discussion at
lease a little.
WRT Knox and defaults:
I have in the past used “do-nothing” implementations as default
placeholders for functionality
that needed extensive per customer configuration, or configuration outside
the
Most of the research I've done around adding Metron as a Knox service is
based on how other projects do it. The documentation is not easy to follow
so I learned by reading other service definition files. The assumption
that we are doing things drastically different is false.
I completely agree
You could say the same thing about Ambari, but that provides mpacks. Knox is
also designed to be extensible through Knox service stacks since they realized
they can’t support every project. The challenge is that the docs have not made
it as easy as they could for the ecosystem to plug into
https://issues.apache.org/jira/browse/KNOX-841?jql=project%20%3D%20KNOX%20AND%20text%20~%20support
Solr is angular for example.
On November 16, 2018 at 08:12:55, Otto Fowler (ottobackwa...@gmail.com)
wrote:
Ok, here is something I don’t understand, but would like to.
Knox comes configured
Ok, here is something I don’t understand, but would like to.
Knox comes configured with build in services for a number of other apache
products and UI’s.
It would seem to me, that the best integration with Knox would be to do
what these other products have done.
1. Do whatever you have to do
Welcome and thanks!
On November 12, 2018 at 10:12:44, Sandeep Moré (moresand...@gmail.com)
wrote:
Hello Ryan,
I am still catching up on the architecture so let me know if I am
misunderstanding anything.
You could have multiple serviced deployed in Knox
1. for metron (metron/api/v1)
2. for
1) Sorry I misspoke. I meant to say this is not possible in the Alerts UI
as far as I know. I put up a PR with a proposed solution here:
https://github.com/apache/metron/pull/1266.
2) Yes Knox is a service you can install with Ambari, similar to Ranger or
Spark. There are some things that are
Thanks Ryan,
1) Can you clarify "not a good way to do this?" Are you saying we don't
have a way to set this and need to add the config option, or that a
solution is not obvious and it's unclear what to do? It seems to me you're
saying the former, but I'd like to be sure.
2) Is Knox not a service
Wanted to give an update on the context path issue. I investigated
rewriting url links in the outgoing static assets with Knox and it was not
trivial. Fortunately I found a simple solution that works with or without
Knox. I changed the base tag in index.html from to , or in other words made
In my view Knox SSO is such a minor feature when it comes to Metron's
capabilities than it's not worth supporting multiple scenarios where it works
with Knox or without Knox. Where we should be configurable (and are
configurable) is on the analytics and stream processing. But this? As long
What you're looking for is an OUT rewrite rule, and a filter rule on
content-type. It's not spectacularly well documented, but
https://knox.apache.org/books/knox-1-0-0/dev-guide.html#Rewrite+Provider
and specifically
https://knox.apache.org/books/knox-1-0-0/dev-guide.html#Rewrite+Steps is a
Hello Ryan,
I am still catching up on the architecture so let me know if I am
misunderstanding anything.
You could have multiple serviced deployed in Knox
1. for metron (metron/api/v1)
2. for alerts-ui (metron-alerts-ui/alerts-list)
and have them run in one Knox instance and you could have
I'm just coming up to speed on Knox so maybe rewriting assets links are
trivial. If anyone has a good example of how to do that or can point to
some documentation, please share.
On Mon, Nov 12, 2018 at 8:54 AM Simon Elliston Ball <
si...@simonellistonball.com> wrote:
> Doing the Knox proxy work
Phew, that was quite the thread to catch up on.
I agree that this should be optional/pluggable to start, and I'm interested
to hear the issues as they relate to upgrading an existing cluster (given
the suggested approach) and exposing both legacy and knox URLs at the same
time.
Jon
On Fri, Nov
A couple more things, and I think this goes without saying - whatever we do
with Knox should NOT
1. Require unit and integration tests to use Knox
2. Break fulldev
Also, I don't know that I saw you mention this, but I'm unsure how we
should leverage Knox as a core piece of the platform.
Thanks for the update Ryan. Per my earlier comments, I thought it might be
the case that we could dramatically simplify this by leveraging Knox's
proxy capabilities, and per your research that appears to be the case. This
is a dramatic simplification and improvement of this feature imo, +1. I'm
I have spent some time recently reviewing this discussion and the feature
branch that Simon put out. I think this is an important feature and want
to move it forward. I started another discussion on adding Knox to our
stack but this discussion has a lot of good context so I will continue it
Apparently, I hit send on my last email before finishing my synopsis (per
@Otto's Q in Slack). To summarize, based on my current understanding I
believe that each of the feature branch changes I've outline above are
units of work that are related, yet should be executed on independently.
Knox SSO
I'm coming in late to the game here, but for my mind a feature branch
should involve the minimum architectural change to accomplish a given
feature.
The feature in question is SSO integration. It seems to me that the
operative question is can we do the feature without making the OTHER
I've spent some more time reading through Simon's response and the added
sequence diagram. This is definitely helpful - thank you Simon.
I need to redact my initial list:
1. Node migrated to Spring Boot, expressjs migrated to a
non-JS/non-NodeJs proxying mechanism (ie Zuul in this case)
Thank you, Simon. The diagrams help a lot
19.09.2018, 21:27, "Simon Elliston Ball" :
> To clarify some of this I've put some documentation into
> https://github.com/apache/metron/pull/1203 under METRON-1755 (
> https://issues.apache.org/jira/browse/METRON-1755). Hopefully the diagrams
> there
To clarify some of this I've put some documentation into
https://github.com/apache/metron/pull/1203 under METRON-1755 (
https://issues.apache.org/jira/browse/METRON-1755). Hopefully the diagrams
there should make it clearer.
Simon
On Tue, 18 Sep 2018 at 14:17, Simon Elliston Ball <
Hi Mike,
Some good points here which could do with some clarification. I suspect the
architecture documentation could be clearer and fill in some of these gaps,
and I'll have a look at working on that and providing some diagrams.
The short version is that the Zuul proxy gateway has been added to
Thanks for the write-up Ryan, this is a great start. I have some further
questions based on your feedback and in addition to my initial thread.
Just for clarification, what version of Knox are we using? HDP 2.6.5, which
is what we currently run full dev against, supports 0.12.0.
I have reviewed a couple different PRs so I'll add some context where I
can. Obviously Simon would be the most qualified to answer but I'll add my
thoughts.
For question 1, while they may not all be necessary I think it does make
sense to include them in this feature branch if our primary goal
Hey all,
I started looking through the Knox SSO feature branch (see here
https://issues.apache.org/jira/browse/METRON-1663). This is some great new
security functionality work and it looks like it will bring some important
new features to the Metron platform. I'm coming at this pretty green, so I
34 matches
Mail list logo
