+1 to that. That’s more the TDE bit, we would also need Kafka SSL, and the Knox stuff (METRON-1663 adds SSL to all the UI and rest api stuff)
Simon > On 15 Aug 2018, at 21:03, Otto Fowler <ottobackwa...@gmail.com> wrote: > > https://issues.apache.org/jira/browse/METRON-106 > At least making sure it is met and closing it > > > >> On August 15, 2018 at 15:53:02, Otto Fowler (ottobackwa...@gmail.com) wrote: >> >> https://issues.apache.org/jira/browse/METRON-343 >> >>> On August 15, 2018 at 15:47:24, Simon Elliston Ball >>> (si...@simonellistonball.com) wrote: >>> >>> What would you see as secure? I’ve seen people use TDE for the HDFS store, >>> but it’s harder to encrypt storage with solr / es. Something I was thinking >>> of doing to follow up on the Knox Feature was to add Ranger integration for >>> securing and auditing configs, and potentially extending to the index >>> destinations. Do you think that would cover the secure storage concept? >>> >>> Simon >>> >>> > On 15 Aug 2018, at 20:39, Otto Fowler <ottobackwa...@gmail.com> wrote: >>> > >>> > Secure storage off the top of my head >>> > >>> > On August 15, 2018 at 14:49:26, zeo...@gmail.com (zeo...@gmail.com) wrote: >>> > >>> > So, as has been discussed in a few >>> > < >>> > https://lists.apache.org/thread.html/0445cd8f94dfb844cd5a23ac3eeca04c9f44c9d8f269c6ef12cb3598@%3Cdev.metron.apache.org%3E> >>> > >>> > other >>> > < >>> > https://lists.apache.org/thread.html/427a20c22207e84331b94e8ead9a4172a22577d26eb581c0e564d0dc@%3Cdev.metron.apache.org%3E> >>> > >>> > recent dev list threads, I would like to discuss what a Metron 1.0 release >>> > looks like. >>> > >>> > In order to kick off the conversation, I would like to make a few >>> > suggestions regarding "what 1.0 means to me," but I'm very interested to >>> > hear everybody else's opinions. >>> > >>> > In order to go 1.0 I believe we should have: >>> > 1. A clear, supported method of upgrading from one version of Metron to >>> > the >>> > next. We have attempted >>> > <https://github.com/apache/metron/blob/master/Upgrading.md> to make this >>> > easier in the past, but it is currently not >>> > < >>> > https://github.com/apache/metron/tree/master/metron-deployment/packaging/ambari/metron-mpack#limitations> >>> > >>> > supported >>> > < >>> > https://github.com/apache/metron/tree/master/metron-deployment/packaging/ambari/elasticsearch-mpack#limitations> >>> > >>> > . >>> > 2. Authentication for all of the UIs and APIs should be secure and support >>> > SSO. I believe this is in progress via METRON-1663 >>> > <https://issues.apache.org/jira/browse/METRON-1663>. >>> > 3. Each of our personas >>> > < >>> > https://cwiki.apache.org/confluence/display/METRON/Metron+User+Personas+And+Benefits> >>> > >>> > should >>> > be well documented, understood, and supported. >>> > - The current state of documentation is, in my opinion, inadequate and I >>> > admit I am partially to blame for this. I suggest we define a strict >>> > approach for documentation, align to it (such as perhaps migrating all >>> > useful wiki documentation to git), and enforce it. >>> > - I would consider METRON-1699 >>> > <https://issues.apache.org/jira/browse/METRON-1699> as a critical item for >>> > a Security Data Scientist, but it is currently not clear to me where the >>> > line exists between some of the other personas, or that each persona has >>> > been sufficiently implemented. >>> > 4. A performance tuning guide should be available for all of the main >>> > components, whether as an independent document or as a part of a larger >>> > document. >>> > 5. Simple data ingest. >>> > - Similar to the ongoing conversation for NiFi integration >>> > < >>> > https://lists.apache.org/thread.html/d7bb4d32c8c42bd40b2f26973f989bcba16010a672fd8a533a5544bf@%3Cdev.metron.apache.org%3E>, >>> > >>> > we should be able to say that we have broken down the barriers to getting >>> > data into a Metron cluster in easy and efficient ways. In addition to >>> > NiFi, having support for other popular tools such as beats >>> > <https://www.elastic.co/products/beats>, fluentd >>> > <https://www.fluentd.org/>, >>> > >>> > etc. >>> > - Parsers should be pluggable, with independent tests and the ability to >>> > make versioned modifications with roll-backs. >>> > >>> > What else? Are any of these items not necessary for a 1.0? >>> > >>> > Jon >>> > -- >>> > >>> > Jon
