FLUME-1531. Flume User Guide should provide more details on configuring the timestamp interceptor.
(Kathleen Ting via Hari Shreedharan) Project: http://git-wip-us.apache.org/repos/asf/flume/repo Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/c49bc323 Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/c49bc323 Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/c49bc323 Branch: refs/heads/cdh-1.2.0+24_intuit Commit: c49bc3237567b203fe4854957f21ba5be85b9b91 Parents: c249b55 Author: Hari Shreedharan <[email protected]> Authored: Fri Aug 31 12:50:04 2012 -0700 Committer: Mike Percy <[email protected]> Committed: Fri Sep 7 14:03:06 2012 -0700 ---------------------------------------------------------------------- flume-ng-doc/sphinx/FlumeUserGuide.rst | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flume/blob/c49bc323/flume-ng-doc/sphinx/FlumeUserGuide.rst ---------------------------------------------------------------------- diff --git a/flume-ng-doc/sphinx/FlumeUserGuide.rst b/flume-ng-doc/sphinx/FlumeUserGuide.rst index 0accbaf..e4720e0 100644 --- a/flume-ng-doc/sphinx/FlumeUserGuide.rst +++ b/flume-ng-doc/sphinx/FlumeUserGuide.rst @@ -1656,17 +1656,22 @@ are named components, here is an example of how they are created through configu .. code-block:: properties agent_foo.sources = source_foo + agent_foo.sinks = hdfs agent_foo.channels = channel-1 agent_foo.sources.source_foo.interceptors = a b agent_foo.sources.source_foo.interceptors.a.type = org.apache.flume.interceptor.HostInterceptor$Builder agent_foo.sources.source_foo.interceptors.a.preserveExisting = false agent_foo.sources.source_foo.interceptors.a.hostHeader = hostname agent_foo.sources.source_foo.interceptors.b.type = org.apache.flume.interceptor.TimestampInterceptor$Builder + agent_foo.sinks.hdfs.filePrefix = FlumeData.%{CollectorHost}.%Y-%m-%d + agent_foo.sinks.hdfs.channel = channel-1 Note that the interceptor builders are passed to the type config parameter. The interceptors are themselves configurable and can be passed configuration values just like they are passed to any other configurable component. In the above example, events are passed to the HostInterceptor first and the events returned by the HostInterceptor -are then passed along to the TimestampInterceptor. +are then passed along to the TimestampInterceptor. You can specify either the fully qualified class name (FQCN) +or the alias ``TIMESTAMP``. If you have multiple collectors writing to the same HDFS path then you could also use +the HostInterceptor. Timestamp Interceptor ~~~~~~~~~~~~~~~~~~~~~ @@ -1678,7 +1683,7 @@ can preserve an existing timestamp if it is already present in the configuration ================ ======= ======================================================================== Property Name Default Description ================ ======= ======================================================================== -**type** -- The component type name, has to be ``TIMESTAMP`` +**type** -- The component type name, has to be ``TIMESTAMP`` or the FQCN preserveExisting false If the timestamp already exists, should it be preserved - true or false ================ ======= ========================================================================
