Jakob Homan created SAMZA-263:
---------------------------------
Summary: Create SystemConsumer and SystemProducer for HDFS
Key: SAMZA-263
URL: https://issues.apache.org/jira/browse/SAMZA-263
Project: Samza
Issue Type: Improvement
Reporter: Jakob Homan
Assignee: Jakob Homan
It would be nice to be able to read/write from HDFS, particularly for
bootstrapping purposes. A few points:
* Per the discussion [about
leveldb|https://issues.apache.org/jira/browse/SAMZA-236?focusedCommentId=13985982&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13985982]
this support should be separated into its own package and project (jar) for
easy testing and severability.
* Similar to the Kafka RegexTopicGenerator, we can enumerate (recursively or
not) the files in an HDFS directory during job startup.
* Connectivity with HCatalog would be interesting as well, but should be
handled in a separate JIRA.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
