Hi Vartika- Thanks for the reminder about the ingestion design document. Suggest we open a JIRA (unless one has already been opened and I missed it) with the specific tasks broken down (create an Epic if necessary) and attach a PDF version of the architecture document. Would also be great to get this document into the Spot repo.
Do we need a separate upstream branch? I don't think there has been much development process discussion here but I would expect the work to be in the main dev branch after the patches/PRs are accepted and the JIRAs are resolved. Michael On Tue, Oct 31, 2017 at 11:31 PM, Vartika Singh <[email protected]> wrote: > Hello all, > > The following document has been out for discussion for a while: > > https://docs.google.com/document/d/1yYaD50gp2HN9RHYaUG8ASDfh3f__ > GyB1Msu2LUofIU4/edit#heading=h.ptz51qlqt0t > > I would like to start a new branch for ingestion component. A > > Again, the idea of ingestion component is the following: > > 1. Extendible to any source. > 2. NRT latency between ingestion and output to the HDFS/persistent > store. > 3. Maintain the integrity of the ODM module in order to facilitate > seamless integration with applications. > > > Thoughts? > > Vartika Singh > -- Michael Ridley <[email protected]> office: (650) 352-1337 mobile: (571) 438-2420 Senior Solutions Architect Cloudera
