Patrick Wendell created FLUME-1507:
--------------------------------------
Summary: Have "Topology Design Considerations" in User Guide
Key: FLUME-1507
URL: https://issues.apache.org/jira/browse/FLUME-1507
Project: Flume
Issue Type: Improvement
Reporter: Patrick Wendell
Assignee: Patrick Wendell
It would be nice if the User Guide had a section which lists considerations for
designing an end-to-end flume topology. I think a lot of people get lost in the
long list of sources/sinks/etc and need a higher level overview of what to
think about when designing a flow. Examples would be:
- When to use Flume? Types of data that flume is good at handling (e.g.
regularly generated, event-based, etc).
- Reliability (explaining that flow reliability is a function of channels used,
redundancy in the flow, and other factors)
- Flume sizing (some basic ideas about how to size the nodes or network you are
running on)
The design space is too large here to give precise recommendations, but just
orienting users that these are the main things they need to be thinking about
would be really helpful. Some issues like reliability are much harder to
explain for Flume-ng than OG, and I think we need documentation making this
explicit. Down the road, a "cookbook" with specific examples would be even
better.
Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
