R P, happy to walk you through https://github.com/DemandCube/Scribengin if your interested
On Wed, Feb 10, 2016 at 5:09 PM, R P <hadoo...@outlook.com> wrote: > Hello All, > New Kafka user here. What is the best way to write Kafka data into HDFS? > I have looked into following options and found that Flume is quickest and > easiest to setup. > > 1. Flume > 2. KaBoom > 3. Kafka Hadoop Loader > 4. Camus -> Gobblin > > Although Flume can result into small file problems when your data is > partitioned and some partitions generate sporadic data. > > What are some best practices and options to write data from Kafka to HDFS? > > Thanks, > R P > > > > -- *Steve Morin | Managing Partner - CTO* *Nvent* O 800-407-1156 ext 803 <800-407-1156;803> | M 347-453-5579 smo...@nventdata.com <smo...@nventdata.com> *Enabling the Data Driven Enterprise* *(Ask us how we can setup scalable open source realtime billion+ event/data collection/analytics infrastructure in weeks)* Service Areas: Management & Strategy Consulting | Data Engineering | Data Science & Visualization BigData Technologies: Hadoop & Ecosystem | NoSql| Hbase | Cassandra | Storm | Spark | Kafka | Mesos | Docker | & More Industries: IoT | Advertising | Retail | Manufacturing | TV & Cable | Energy | Oil & Gas | Insurance | Finance | Telecom
