All, Keeping in mind the scope of growth in this project, we have come with this initial design document for ingestion. The components of the ingestion module have been designed keeping in mind the following constraints:
1. Extendible to any source. 2. No latency between ingestion and output to the HDFS/persistent store. 3. Maintain the integrity of the ODM module in order to facilitate seamless integration with applications. We are opening this document for review and all comments, feedbacks and suggestions will be very welcome and will be helpful. https://docs.google.com/document/d/1yYaD50gp2HN9RHYaUG8ASDfh3f__GyB1Msu2LUofIU4/edit#heading=h.ptz51qlqt0t Vartika Singh
