Re: [DISCUSS] Persistence store for user profile settings

2018-02-12 Thread Michael Miklavcic
I'm also good with HBase. On Fri, Feb 9, 2018 at 2:14 PM, Nick Allen wrote: > +1 I think going with HBase is a good approach for now. Thanks for laying > out the pros and cons. > > On Fri, Feb 9, 2018 at 3:46 PM, Ryan Merriman wrote: > > > I would like to bring this discussion to a conclusion

Re: [DISCUSS] Persistence store for user profile settings

2018-02-09 Thread Nick Allen
+1 I think going with HBase is a good approach for now. Thanks for laying out the pros and cons. On Fri, Feb 9, 2018 at 3:46 PM, Ryan Merriman wrote: > I would like to bring this discussion to a conclusion and update the PR > accordingly. To clarify on whether we depend on an RDBMS right now,

Re: [DISCUSS] Persistence store for user profile settings

2018-02-09 Thread Ryan Merriman
I would like to bring this discussion to a conclusion and update the PR accordingly. To clarify on whether we depend on an RDBMS right now, we do but only for authentication which will probably be replaced at some point. So the answer is not really. I personally agree with Simon and think we shou

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Nick Allen
> Glad you agree with me that this isn’t HBase scale… it’s clearly not. I would never suggest introducing HBase for something like this, but since it’s there. Ah, gotcha. Misunderstood your statement. On Fri, Feb 2, 2018 at 9:01 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > G

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
Glad you agree with me that this isn’t HBase scale… it’s clearly not. I would never suggest introducing HBase for something like this, but since it’s there. On the idea of using the Ambari RDBMS for the same basis of it being there, I see your point. That said, it can be postgres, sql server, my

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Nick Allen
I fall marginally on the side of an RDBMS. There is definitely a case to be made on both sides, but I'll point out a few things for the RDBMS. (1) Flexibility. Using an RDBMS is going to provide us with much greater flexibility going forward. We really don't know what the specific use cases wi

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
Couldn’t agree with you more Otto! On the perms / ACLs / AXOs / groups / users etc concerns though, there are other Apache projects (such as Ranger) which have already done a lot of the hard thinking and architecture / data structure / admin ui and persistence pieces for us, so I’d say we lean o

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Otto Fowler
Fair enough, I don’t have a preference. I think my point is that we need to understand the use cases we can think of more, especially if we are going to be having permissions, grouping and crud around that, and preloading, before just throwing everything in RDBMS -or- HBASE. On February 2, 201

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
True, and that is a requirement I’ve heard a lot (standard views or field sets in shared sets of saved search for example). That would definitely rule out sticking with the current approach (browser local storage, per Casey’s suggestion below). That said, I’m not sure that changes my views on

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Otto Fowler
It is not uncommon to want to have ‘shared’ preferences or setups. Think of shared dashboards or queries vs. personal version in jira. Would RDBMS help with that? On February 2, 2018 at 07:17:04, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Introducing a RDBMS to the stack seems

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
Introducing a RDBMS to the stack seems unnecessary for this. If we consider the data access patterns for user profiles, we are unlikely to query into them, or indeed do anything other than look them up, or write them out by a username key. To that end, using an ORM to translate a a nested config

Re: [DISCUSS] Persistence store for user profile settings

2018-02-01 Thread Michael Miklavcic
Personally, I'd be in favor of something like Maria DB as an open source repo. Or any other ansi sql store. On the positive side, it should mesh seamlessly with ORM tools. And the schema for this should be pretty vanilla, I'd imagine. I might even consider skipping ORM for straight JDBC and simple

Re: [DISCUSS] Persistence store for user profile settings

2018-02-01 Thread Casey Stella
So, I'll answer your question with some questions: - No matter the data store we use upgrading will take some care, right? - Do we currently depend on a RDBMS anywhere? I want to say that we do in the REST layer already, right? - If we don't use a RDBMs, what's the other option? What

[DISCUSS] Persistence store for user profile settings

2018-02-01 Thread Ryan Merriman
There is currently a PR up for review that allows a user to configure and save the list of facet fields that appear in the left column of the Alerts UI: https://github.com/apache/metron/pull/853. The REST layer has ORM support which means we can store those in a relational database. However I'm