Hello Sam, The design could easily be extended to a several streaming cases.
Adding more talent pool for development is always a good idea. As long as there is active development and contribution. Once the design goes through, and if we can get a few committers (hence growing the community) actively developing on the project, it would definitely boost the growth and turnaround time overall. We have been a bit slow on that. Thank you Vartika On Wed, Oct 4, 2017 at 9:15 PM, Samuel Heywood <sam.heyw...@cloudera.com> wrote: > Vartika > > I'm reading through the ingest doc. This is very good. > > Question for you - this ingest design, it is extensible to use cases > outside of cyber right? I mean, if we wanted to tap into online e-Commerce > data sources we could still use the core aspects of your design right? > > I'm asking as if that's true, it may make it more possible to unlock > resources within Cloudera to help develop this stuff > > Thank you Vartika, > > Sam > > On Thu, Oct 5, 2017 at 10:38 AM, Vartika Singh <vsi...@cloudera.com> > wrote: > > > All, > > > > Keeping in mind the scope of growth in this project, we have come with > this > > initial design document for ingestion. The components of the ingestion > > module have been designed keeping in mind the following constraints: > > > > 1. Extendible to any source. > > 2. No latency between ingestion and output to the HDFS/persistent > store. > > 3. Maintain the integrity of the ODM module in order to facilitate > > seamless integration with applications. > > > > We are opening this document for review and all comments, feedbacks and > > suggestions will be very welcome and will be helpful. > > > > https://docs.google.com/document/d/1yYaD50gp2HN9RHYaUG8ASDfh3f__ > > GyB1Msu2LUofIU4/edit#heading=h.ptz51qlqt0t > > > > > > Vartika Singh > > > > > > -- > Sam Heywood > Director Cybersecurity Strategy, Cloudera > sam.heyw...@cloudera.com <sam.heyw...@gazzang.com> > M: (512) 716-9660 > -- Vartika Singh Cloudera
