Re: [DISCUSS] Persisting user data

2017-08-03 Thread Ryan Merriman
Spring is JDBC-generic so I think we're good there. Improving our docs on this topic is being discussed in https://github.com/apache/metron/pull/646 so hopefully this will be clear once that's worked out. Simon is correct, I found out the hard way that Hibernate is not an option because of it's l

Re: [DISCUSS] Persisting user data

2017-08-03 Thread Simon Elliston Ball
Anything spring based is likely multi-db by definition as long as a we pick a good friendly ORM (not hibernate because licensing problems with apache, eclipselink?) But I suspect we should pick a good default and that that default should be postgres. > On 3 Aug 2017, at 10:24, Casey Stella wr

Re: [DISCUSS] Persisting user data

2017-08-03 Thread Casey Stella
I'd vote for a DB-based solution, but I'd argue that any solution shouldn't be database specific (i.e. postgres), but JDBC-generic. People and organizations have very strong views regarding databases and I'd prefer to side-step those holy wars by being agnostic. On Wed, Aug 2, 2017 at 9:36 PM, Ry

Re: [DISCUSS] Persisting user data

2017-08-02 Thread Ryan Merriman
Spring supports a variety of databases including Postgres. I have no problem with using Postgres instead of MySQL. On Wed, Aug 2, 2017 at 3:32 PM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > Agreed on Postgres. It's a lot easier to work with license-wise in apache > projects, and

Re: [DISCUSS] Persisting user data

2017-08-02 Thread Simon Elliston Ball
Agreed on Postgres. It's a lot easier to work with license-wise in apache projects, and has a lot of the capability we need here, especially if we can find a sensible ORM. Anyone got any thoughts on what would work there? Simon > On 2 Aug 2017, at 21:21, Matt Foley wrote: > > Hi Ryan, > Zook

Re: [DISCUSS] Persisting user data

2017-08-02 Thread Matt Foley
Hi Ryan, Zookeeper has a default (and seldom changed) max znode size of 1MB, but it is “designed to store data on the order of kilobytes in size.”[1] And it’s not really intended for frequently-changing data, which is okay here. But I just included it for completeness, I’m not advocating for i

Re: [DISCUSS] Persisting user data

2017-08-02 Thread Ryan Merriman
Matt, Thank you for the suggestions. I forgot to include Zookeeper. Are there any tradeoffs we should be aware of if we decide to use Zookeeper? Are there guidelines for how much data can be stored in Zookeeper? To answer your questions: 1. I think both use cases make sense so a combination

Re: [DISCUSS] Persisting user data

2017-07-26 Thread Matt Foley
There’s a couple other places you could put config info (but maybe not saved searches): - Zookeeper - metron-alerts-ui/config.xml or config.json file - the Ambari database, whichever it happens to be Questions that influence the decision include: 1. Should there be one configuration shared am

[DISCUSS] Persisting user data

2017-07-26 Thread Ryan Merriman
In anticipation of METRON-988 being merged into master, there will be a need to persist user preferences such as UI layout, saved searches, search history, etc. I think where and how we persist this data should be discussed in order to facilitate a design. This data won't be large in scale and ma