Great feature to move to LDAP integration and hopefully Ranger
integration afterwards. Does it need to support LDAP and AD separately?

Cheers,
Ali

On Sat, Nov 17, 2018 at 3:29 AM Otto Fowler <ottobackwa...@gmail.com> wrote:

> I would like to understand the work required to move our JDBC support ( or
> adapt the current support to the abstraction ) to /contrib.
> We could default and only officially support LDAP, but have the /contrib (
> or /extension_examples ) have a “this is how you would support jdbc for
> auth “ project.
>
>
>
> On November 15, 2018 at 15:01:10, Michael Miklavcic (
> michael.miklav...@gmail.com) wrote:
>
> Yes, makes sense. +1 to that.
>
> On Thu, Nov 15, 2018 at 12:54 PM James Sirota <jsir...@apache.org> wrote:
>
> > To clarify my position, I don't have a problem with mySql or any other
> > projects relying on it. mySql in itself is not an issue. What I don't
> > want is for a customer to be presented with an option to chose and
> > configure two options for authenticating the UI, which I think is
> > needless. It adds complexity for not much value. Since LDAP is clearly
> > the better way to go that should be what we support without explicitly
> > giving a user an option to switch to JDBC. A user can still do so by
> > extending our abstractions if that is what they chose to do, but this
> would
> > not be officially supported by us. We would not be providing a config or
> > an mPack to do this. A user would have to do it on their own.
> >
> > James
> >
> >
> >
> > 15.11.2018, 12:15, "Michael Miklavcic" <michael.miklav...@gmail.com>:
> > > Incidentally, even without the Metron piece in the picture, what is the
> > > answer for Ambari's database dependency? Which uses a SQL data store.
> > Does
> > > this actually solve the problem of "customers won't install Metron bc
> SQL
> > > store?" or are there other issues we need to address?
> > >
> > > On Thu, Nov 15, 2018 at 9:30 AM James Sirota <jsir...@apache.org>
> wrote:
> > >
> > >> Hi Guys,
> > >>
> > >> My opinion on this, as is with Knox SSO, is that the code should be
> > >> pluggable to support JDBC, but we should not continue to support the
> > >> concrete implementation and expose it to users via a setting. This is
> a
> > >> fairly minor feature and the added complexity of supporting switching
> > >> between JDBC and LDAP is simply not worth it. We need to strike a
> > balance
> > >> between ease of use and capabilities/extensibility. For features that
> > are
> > >> worth it such as with analytics and stream processing, the extra
> > capability
> > >> is worth the added complexity in configuration. But for this, it is
> > not.
> > >> So let's keep JDBC around for a release to allow users to migrate to
> > LDAP,
> > >> deprecate it, and move on.
> > >>
> > >> Thanks,
> > >> James
> > >>
> > >> 13.11.2018, 16:03, "Simon Elliston Ball" <si...@simonellistonball.com
> > >:
> > >> > We went over the hbase user settings thing on extensive discussions
> > at
> > >> the time. Storing an arbitrary blob of JSON which is only ever
> > accessed by
> > >> a single key (username) was concluded to be a key value problem, not a
> > >> relational problem. Hbase was concluded to be massive overkill as a
> key
> > >> value store in this usecase, unless it was already there and ready to
> > go,
> > >> which in the case of Metron, it is, for enrichments, threat intel and
> > >> profiles. Hence it ended up in Hbase, as a conveniently present data
> > store
> > >> that matched the usage patterns. See
> > >>
> >
>
> https://lists.apache.org/thread.html/145b3b8ffd8c3aa5bbfc3b93f550fc67e71737819b19bc525a2f2ce2@%3Cdev.metron.apache.org%3E
> > >> and METRON-1337 for discussion.
> > >> >
> > >> > Simon
> > >> >
> > >> >> On 13 Nov 2018, at 18:50, Michael Miklavcic <
> > >> michael.miklav...@gmail.com> wrote:
> > >> >>
> > >> >> Thanks for the write up Simon. I don't think I see any major
> > problems
> > >> with
> > >> >> deprecating the general sql store. However, just to clarify, Metron
> > >> does
> > >> >> NOT require any specific backing store. It's 100% JPA, which means
> > >> anything
> > >> >> that can be configured with the Spring properties we expose. I
> think
> > >> the
> > >> >> most opinionated thing we do there is ship an extremely basic table
> > >> >> creation script for h2 and mysql as a simple example for schema. As
> > an
> > >> >> example, we simply use H2 in full dev, which is entirely in-memory
> > and
> > >> spun
> > >> >> up automatically from configuration. The recent work by Justin Leet
> > >> removes
> > >> >> the need to use a SQL store at all if you choose LDAP -
> > >> >> https://github.com/apache/metron/pull/1246. I'll let him comment
> > >> further on
> > >> >> this, but I think there is one small change that could be made via
> a
> > >> toggle
> > >> >> in Ambari that would even eliminate the user from seeing JDBC
> > settings
> > >> >> altogether during install if they choose LDAP. Again, I think I'm
> on
> > >> board
> > >> >> with deprecating the SQL backing store as I pointed this out on the
> > >> Knox
> > >> >> thread as well, but I just wanted to make sure everyone has an
> > accurate
> > >> >> picture of the current state.
> > >> >>
> > >> >> I had to double check on the HBase config you mentioned, but it
> does
> > >> appear
> > >> >> that we use it for the Alerts UI. I don't think I realized we were
> > >> storing
> > >> >> config there instead of the Zookeeper store we use for other system
> > >> >> configuration. Ironically enough, I think that it probably makes
> > more
> > >> sense
> > >> >> than the current auth info to store in a traditional sql store,
> > however
> > >> >> it's in HBase currently so it's a non-issue wrt SQL/JPA either way,
> > as
> > >> you
> > >> >> pointed out.
> > >> >>
> > >> >> Whatever architectural changes we choose to add here, I think we
> > need
> > >> to
> > >> >> emphasize pluggability regardless of the specific implementation.
> > That
> > >> is
> > >> >> to say, I don't think we should make a hard requirement on Knox, in
> > >> order
> > >> >> to get LDAP, in order to deprecate an optional general SQL backing
> > >> store.
> > >> >> It makes sensible defaults if that's where we want to go, which is
> > the
> > >> way
> > >> >> we have done things for most of the successful features I've seen
> in
> > >> >> Metron. Provide all the options should a user desire them, but
> > abstract
> > >> >> away the complexity in the UIs.
> > >> >>
> > >> >> Best,
> > >> >> Mike
> > >> >>
> > >> >> On Tue, Nov 13, 2018 at 5:42 AM Simon Elliston Ball <
> > >> >> si...@simonellistonball.com> wrote:
> > >> >>
> > >> >>> I've been coming across a number of organisations who are blocked
> > from
> > >> >>> installing Metron by the MySQL auth database.
> > >> >>>
> > >> >>> The main problems with our MySQL default are:
> > >> >>>
> > >> >>> * What? Un-ecrypted passwords?!? - which frankly is embarrassing
> > in a
> > >> >>> security platform and usually where the deployment conversation
> > ends
> > >> for me
> > >> >>> * MySQL install varies from platform to platform
> > >> >>> * An additional database to manage, backup, etc. so now I have to
> > >> talk to a
> > >> >>> DBA
> > >> >>> * Harder to maintain HA for this without externalising and
> fighting
> > >> against
> > >> >>> our defaults
> > >> >>> * There are a lot of dependencies for just storing a table of
> users
> > >> >>> (Eclipse Link, JPA, the MySQL server and the need to get clients
> > >> installed
> > >> >>> and pushed separately because of licence requirements)
> > >> >>> * Organisations don't want to have to manage yet another user
> > source
> > >> of
> > >> >>> truth since this leads to operational complexity.
> > >> >>>
> > >> >>> In short, managing our own user store makes very little sense to
> > >> operations
> > >> >>> users.
> > >> >>>
> > >> >>> Some of these (licence and inconsistency for example) could be
> > solved
> > >> by
> > >> >>> changing our default DB to something like Postgres, which has
> > easier
> > >> terms
> > >> >>> to deal with. We could start encrypting passwords, but there would
> > >> still be
> > >> >>> a lot of dependencies to store users, which is a problem much
> > better
> > >> solved
> > >> >>> by LDAP.
> > >> >>>
> > >> >>> Now that we have the option to use LDAP for user storage, I would
> > >> suggest
> > >> >>> that we deprecate and ultimately remove all the RDBMS and ORM
> > >> dependencies,
> > >> >>> which significantly reduces our dependencies and simplifies
> > >> deployment and
> > >> >>> long term management of Metron clusters.
> > >> >>>
> > >> >>> So I propose that we deprecate the RDBMS use in the next Apache
> > >> release,
> > >> >>> and then strip out the RDBMS stuff in the following. We would
> > >> continue to
> > >> >>> use LDAP for users and HBase for non-LDAPy user settings (as we
> > >> currently
> > >> >>> do). We should also provide a small demo LDAP for full dev. Since
> > we
> > >> are
> > >> >>> looking at adding Knox into the stack, that project provides a
> > >> convenient
> > >> >>> mini-LDAP demo service which would do this job without the need to
> > add
> > >> >>> additional components.
> > >> >>>
> > >> >>> Thoughts? Anyone relying on MySQL for users (if so, are you aware
> > >> that your
> > >> >>> passwords are all plaintext? How do you currently handle the
> > >> shortcomings
> > >> >>> and admin overhead?) Any objections?
> > >> >>>
> > >> >>> Simon
> > >>
> > >> -------------------
> > >> Thank you,
> > >>
> > >> James Sirota
> > >> PMC- Apache Metron
> > >> jsirota AT apache DOT org
> >
> > -------------------
> > Thank you,
> >
> > James Sirota
> > PMC- Apache Metron
> > jsirota AT apache DOT org
> >
> >
>


-- 
A.Nazemian

Reply via email to