Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The following page has been changed by AriRabkin: http://wiki.apache.org/hadoop/Chukwa_Adaptors_List The comment on the change is: document adaptors more clearly. ------------------------------------------------------------------------------ - The following adaptors are built into Chukwa and can be added to the agent using the command line provided after each. These lines can also be added to the conf/initial_adaptors file to have agents run adaptors at startup time. + In Chukwa, data is collected by adaptors. Adaptors run inside the agent process, and can be dynamically started and stopped. - In order to start an adaptor on an agent, you can telnet into the agent (i.e. and give the "ADD" command line to the agent. + (mention that commands can go in initial_adaptors or else at control port) - '''!FileTailer''' + The general syntax of the add command is as follows: - '''!FileTailingAdaptor''' + {{{add [name =] <adaptor_class_name> <datatype> <adaptor specific params> <initial offset>.}}} + + The `name` field lets you give this adaptor a meaningful name, in order to stop it later. If you do not specify a name, Chukwa will autogenerate one by hashing the class name and parameters. The adaptor class name specifies what adaptor to run. The `datatype` specifies how to process the collected data. The initial offset should normally be zero; specifying a nonzero initial offset lets you do advanced things like only processing a portion of a file. - '''CharFileTailingAdaptorUTF8''' + The adaptors built into Chukwa are as follows: - '''CharFileTailingAdaptorUT!F8NewLineEscaped''' add org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUT!F8NewLineEscaped SysLog 0 /var/log/messages 0 + '''!FileAdaptor''': Push a whole file, as one Chunk, then exit. Requires file name as parameter. - '''!TerminatorThread''' + Example: {{{add FileTailer }}} + '''!filetailer.FileTailingAdaptor''': The + Pushes the file as flat bytes, ignoring the content. Useful for streaming binary data. + '''!filetailer.CharFileTailingAdaptorUTF8''' + The same, but breaking chunks only at carriage returns. Useful for most ASCII log file formats. + '''!filetailer.CharFileTailingAdaptorUT!F8NewLineEscaped''' + The same, but breaking chunks only at non-escaped carriage returns. Useful for pushing Chukwa-formatted log files, where exception stack traces stay in a single chunk. +
